Algorithmic Bias Examples

    One of the most famous cases of algorithmic bias at the corporate level was the case of Amazon’s AI recruiting tool. Reuters reported that Amazon had developed an AI-developed tool to filter job candidates, but it had to be shut down due to bias found in the tool: “machine-learning specialists uncovered a big problem: their new recruiting engine did not like women (…) computer models were trained to vet applicants by observing patterns in resumes submitted to the company over a 10-year period. Most came from men, a reflection of male dominance across the tech industry” (Dastin, 2018). Amazon did not have a technical problem, but rather a conceptual one. Artificial Intelligence is only as good as the baseline data it is sourced with, which can be tainted with many human biases. As more software-based alternatives to complex human processes that require processing data emerge, smaller companies than Amazon will also start using software-based intelligence to make certain decisions. It is concerning that when this happens, issues like the one experienced by Amazon’s tool will multiply across several industries without proper prevention. There is a need to provide standards or create regulations to prevent algorithms and specially machine learning produced algorithms from skipping the already existing laws such as anti-discriminatory legislation.

    The medical industry has also been using AI. One well-known application is in imaging screening. Scientific American reported that: “algorithms trained with gender-imbalanced data do worse at reading chest x-rays for an underrepresented gender, and researchers are already concerned that skin-cancer detection algorithms, many of which are trained primarily on light-skinned individuals, do worse at detecting skin cancer affecting darker skin” (Kaushal, 2020). Once again, in this case, the cause of the bias is in the quality of the datasets used to train the AI.

    Another example of bias was the UnitedHealth Group’s algorithm that was allegedly racially biased and led to lower level of care for black patients. NBC News reported that “the algorithm determined that black patients spent $1,800 less in medical costs per year than white patients with the same chronic conditions, leading the algorithm to conclude incorrectly that the black patients must be healthier since they spend less on health care” (Gawronski, 2019). I am embedding a video of the interview CBS made to Melanie Evans (CBS News, 2019), a Wall Street Journal reporter who wrote a piece on this topic (Evans & Mathews, 2019). In this interview, Evans discusses also some good points about why bias in algorithms are a concern.


New York Regulator Probes UnitedHealth Algorithm for Racial Bias (Source: CBS News)

References

Dastin, J. (2018, October 11). Amazon scraps secret AI recruiting tool that showed bias against women. U.S. https://www.reuters.com/article/us-amazon-com-jobs-automation-insight/amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G

Kaushal, A. R. A. (2020, November 17). Health Care AI Systems Are Biased. Scientific American. https://www.scientificamerican.com/article/health-care-ai-systems-are-biased/

Gawronski, Q. (2019, November 7). Racial bias found in widely used health care algorithm. NBC News. https://www.nbcnews.com/news/nbcblk/racial-bias-found-widely-used-health-care-algorithm-n1076436

Evans, M., & Mathews, A. W. (2019, October 26). New York Regulator Probes UnitedHealth Algorithm for Racial Bias. WSJ. https://www.wsj.com/articles/new-york-regulator-probes-unitedhealth-algorithm-for-racial-bias-11572087601

CBS News. (2019, November 1). Racial bias found in health care company algorithm [Video]. YouTube. https://www.youtube.com/watch?v=KYYVjT0mQB8&feature=youtu.be

Comments

Post a Comment

Popular posts from this blog

Could the FTC Order to Delete Biased Algorithms in the Near Future?

Image
Continuing with my endeavor to further understand algorithmic bias, as well as learning about what is being done about it, I recently stumbled upon a fascinating article in Protocol that talked about how the Federal Trade Commission (FTC) had instructed the company WW International to delete all the illegally obtained private data and delete all algorithms derived from this data. The subject was intriguing, “a new standard for penalizing tech companies that violate privacy and use deceptive data practices: algorithmic destruction” (Kaye, 2022). Started looking further into the details by reading the FTC’s press release on the settlement resulting from a complaint filed by the Department of Justice (DOJ) on behalf of the FTC on WW International (formerly Weight Watchers). The FTC press release indicated that WW International collected personal information from children as young as eight without parental permission. The settlement instructed not only for the illegally collected data to b
Read more

Algorithmic Bias: Is Perfectly Imperfect Good Enough?

Image
A Forbes article titled “Perfectly Imperfect: Coping With The ‘Flaws’ Of Artificial Intelligence (AI)” claims that AI will never be perfect and that bias should be expected and accepted by society. That implicit bias, poor data, and people expectations make the case for AI to never be perfect. It goes beyond that and claims that AI is perfectly imperfect (Sahota, 2020). Nevertheless, the article’s points of view have one common flaw, the unparalleled size and speed at which AI can operate. As mentioned in an article published by McKinsey and Company on bias in AI: “extensive evidence suggests that AI models can embed human and societal biases and deploy them at scale” (Silberg & Manyika, 2020). XKCD (AI Hiring Algorithm, n.d.) makes a great example of this in Figure 1.   Figure 1 . AI Hiring Algorithm. (Source: xkcd.com,  n.d.)   Although it is logical to understand that no creation can be perfect and that all we can do is to limit bias and other issues with AI to its minimum, we m
Read more

AI Bias: Human Bias in Artificial-Intelligence-Developed Algorithms

Image
Wondering how smart assistants like Siri or Alexa keep getting better at understanding voice instructions over time, or how Netflix seems to keep getting better at suggesting content? Or perhaps you wonder sometimes how you are getting all these ads about something you never searched for but was exactly what you needed? These are only some of the most visible and least critical interactions we have with Artificial-Intelligence-developed algorithms every day of our lives. They are designed to learn from us, analyze our interests and decision-making thinking, and ultimately outsmart us. The Wall Street Journal has a very interesting article describing how the use of AI is expanding: “We are witnessing a turning point for artificial intelligence, as more of it comes down from the clouds and into our smartphones and automobiles (…) Shield AI, a contractor for the Department of Defense, has put a great deal of AI into quadcopter-style drones which have already carried out—and continue to be
Read more