Our Algorithms are Biased
Who will you meet?
Cities are innovating, companies are pivoting, and start-ups are growing. Like you, every urban practitioner has a remarkable story of insight and challenge from the past year.
Meet these peers and discuss the future of cities in the new Meeting of the Minds Executive Cohort Program. Replace boring virtual summits with facilitated, online, small-group discussions where you can make real connections with extraordinary, like-minded people.
Algorithms are enticing. Algorithms are fascinating. Algorithms are the “new and cool” tool for many of us. And algorithms can do a world of good. But algorithms can also be problematic.
Let’s be honest with ourselves.
All people have bias, all data have bias, therefore all algorithms have bias. That is the truth. If we are not perfect, then neither can be our creations. The sooner we all congregate around this truth, the sooner we might stop seeing shocking news stories like “Amazon’s hiring algorithm is sexist”, or “Google face recognition algorithm tags black people as gorillas”, or “controversial machine learning algorithm disparately sends black offenders to prison.” Algorithms are mirrors, reflecting the truths and inequalities that they recognize within our data. All they do is display them back to us; why are we shocked when we see them? So, yes, your (yes, you) algorithms are biased. Let’s just start there.
Our creations are not without flaw. Despite being advertised as entities that “simplify” and make processes more efficient, algorithms are still complex. They are nuanced, and should always require injections of purposeful thoughtfulness and extreme caution. They are complicated. They all have their issues. If we all admit this to one another—breaking down personal feelings of guilt or shame, and then together stand in these truths, we can all begin working towards solutions.
Let’s all say this together: All algorithms start out imperfect. That doesn’t mean they have to stay that way. Now, let’s get to work.
Something that can help.
Recently, a multidisciplinary, bi-coastal team (of which I am a part) created and debuted a tool that can help ameliorate the sometimes-problematic consequences that algorithms pose, called The Ethics & Algorithms Toolkit. For our team, the most feasible, helpful, and active solution for the issues that algorithms pose is risk assessment and mitigation. The risks are there, what are you going to do to address them?
Our work on algorithms
Like I’ve mentioned, our team just crafted a new tool that focuses on integrating several layers of ethical consideration into the process of evaluating algorithms. We are a group of passionate individuals from all different walks of life and career paths with a common interest: Fighting for better quality of life and outcomes for all. In order to do this in the context of algorithms, we decided to create a tool that bridges the gap between experienced data scientists and your average government practitioner. The challenge we faced was to create an interactive, intuitive risk scoring tool, that would be complex enough to address advanced data science concepts yet simple enough to not overwhelm the reader. And a tool that started as a lengthy, clunky, formal document transformed into a shortened, color-coded, plug-and-play tool for all.
The toolkit’s team is comprised of myself, a data analyst at the Center for Government Excellence (GovEx) and recent graduate from the Quantitative Methods in the Social Sciences (QMSS) program at Columbia University, Andrew Nicklin, director of Data Practices at GovEx, Joy Bonaguro, former Chief Data Officer in the City of San Francisco and current Director of People, Operations and Data at Corelight, Dave Anderson, advisory board member at Data Community DC, and Jane Wiseman, senior fellow at the Ash Center for Democratic Governance and Innovation at Harvard University. Moving forward, our friend Mo Johnson, Project Lead of the Global Data Ethics Project at Data for Democracy, will also be adding in her expertise. But enough about us, let’s get back to the toolkit.
The Ethics & Algorithms Toolkit
For almost a year, our team has been working on a toolkit to help readers navigate the nuanced, complicated conversations that surround algorithms and the data that they consume. The project came about after a small workshop held in the city of San Francisco in February of 2018. The conversation around data science and transparency for laypeople brought us to the idea that a new resource was needed to bridge the gap between data scientists and non-data scientists. Today, our tool exists in several parts that can and should be used in conjunction. We begin with a hearty background to get readers acclimated to terms like “black box” and “reinforcement learning” and our structure, then continue through our risk assessment (which comes with a handy worksheet) and risk mitigation sections, and conclude with appendices.
Beta version of our toolkit then received its official debut at D4GX (Data for Good Exchange held at Bloomberg HQ) in September.
Real use cases.
A large mid-Atlantic city (for privacy reasons, we cannot yet divulge which city) recently held an internal data science meeting to discuss the toolkit, and plans to use it for two upcoming data science projects: one around a housing initiative, and one around a public health initiative. During this meeting, I noticed how curious people were about how they could use our tool. I also noticed how much the toolkit sparked different paths for conversation, which is our intention. We want people to get into new, invigorating conversations with their colleagues about algorithms that they are using or want to use. Two other U.S. cities have contacted us personally to let us know that they have forked our work and are using it to structure data science projects. Conversations around the toolkit have been energetic and complimentary, with many people saying that this tool has reached them at the perfect time. We are excited to know that our work is resonating with so many.
We’ve also received a tremendous amount of positive feedback in 2018 at events like MetroLab, DataCon, and internal city meetings that GovEx has attended. In 2019, we hope to continue to gather feedback and partner with cities using the toolkit to work through live projects and evaluate algorithms.
Where are we headed next?
Looking to the future, we are really excited to see the unique and intimate ways in which our tool is used across the country. We hope to continue to build upon our work and have many ideas for how to make this tool even better. In order to do this, we will continue conversations with our partners, former colleagues, and friends around the country. Additionally, GovEx plans to integrate this toolkit into training coursework in 2019. To see our training offerings, you can visit https://govex.academy.
Leave your comment below, or reply to others.
Please note that this comment section is for thoughtful, on-topic discussions. Admin approval is required for all comments. Your comment may be edited if it contains grammatical errors. Low effort, self-promotional, or impolite comments will be deleted.
Read more from MeetingoftheMinds.org
Spotlighting innovations in urban sustainability and connected technology
Since the Great Recession of 2008, the housing wealth gap has expanded to include not just Black and Brown Americans, but younger White Americans as well. Millennials and Generation Z Whites are now joining their Black and Brown peers in facing untenable housing precarity and blocked access to wealth. With wages stuck at 1980 levels and housing prices at least double (in inflation adjusted terms) what they were 40 years ago, many younger Americans, most with college degrees, are giving up on buying a home and even struggle to rent apartments suitable for raising a family.
What makes it hard for policy people and citizens to accept this truth is that we have not seen this problem in a very long time. Back in the 1920s of course, but not really since then. But this is actually an old problem that has come back to haunt us; a problem first articulated by Adam Smith in the 1700s.
More than ever, urban transit services are in need of sustainable and affordable solutions to better serve all members of our diverse communities, not least among them, those that are traditionally car-dependent. New mobility technologies can be a potential resource for local transit agencies to augment multi-modal connectivity across existing transit infrastructures.
We envision a new decentralized and distributed model that provides multi-modal access through nimble and flexible multi-modal Transit Districts, rather than through traditional, centralized, and often too expensive Multi-modal Transit Hubs. Working in collaboration with existing agencies, new micro-mobility technologies could provide greater and seamless access to existing transit infrastructure, while maximizing the potential of the public realm, creating an experience that many could enjoy beyond just catching the next bus or finding a scooter. So how would we go about it?
Dedicated anti-trafficking actors across the nation are trying to build better systems in big jurisdictions like New York, San Francisco, and Los Angeles, and in smaller but scrappy jurisdictions like Waco, Texas and Boaz, Alabama. They all share the same need, for stronger interconnectedness as an anti-trafficking field, and more collaboration.
The Forging Freedom Portal is a one-stop shop where a police officer planning a victim-centered operation can connect with their law enforcement counterparts, and the right service providers ahead of time, collaborating to make sure they’re planning for the language skills, social services, and legal support that victims may need. The portal is a place where the people who care most about ending human trafficking, who are doing the hard work every day on the ground, can learn from each other and share best practices to raise the collective standard of this work.