Red Flag 5. Algorithmic decision-making that can result in discrimination

This Red Flag may appear in a company’s value proposition.

Banks, insurance firms and mortgage companies using automated decision-making that results in individuals being declined credit based on their age or race
Recruitment companies using algorithmic systems to help employers make decisions about candidates in ways that result in certain groups such as women and ethnic minorities being disproportionality removed from recruitment processes
Companies offering algorithmic tools and solutions to public institutions that make decisions about:
- Law enforcement (e.g., for use in predicting crimes based on ethnicity)
- Immigration (e.g., for use in group profiling and prediction used in detention, deportation or onitoring decision-making)
- Social scoring systems (e.g., for use in determining access to services, benefits or opportunities)
Social media companies selling profiling and targeting services that enable political campaigners to spread misinformation about opponents or election dates in ways that undermine the ability of individuals to participate in political processes without interference

Companies that offer or make use of targeted advertising such as social media, search, websites, blogs, as well as advertisers and their media agencies
Consumer finance and credit
Health insurance and healthcare
Retailers when targeting customers with product promotions
Recruitment industry including online portals and software providers
IT and technology companies selling algorithmic solutions to government agencies including healthcare and the criminal justice system
Political consulting firms
Data brokers selling data analytics as a service