Computing is ubiquitous in our daily lives. Increasingly computing makes decisions in our lives starting from the mundane (e.g. Netflix recommending us movies/TV shows or sensors dispensing soap), to the somewhat more relevant (e.g. algorithms deciding which ads Google shows you or a car parking itself) to the downright worrisome (e.g. automated systems deciding the risk of a person who is arrested committing a crime in the future or automated “killer drones”).
While we have (and continue to) educate our computing students on the technical aspects of building computing systems that are used in our daily lives, there has been relatively less focus on incorporating ethics and responsible computing in such courses. Our project aims to incorporate ethics and more generally responsible computing in a wide range of computing courses.
While most of our work in this project has been in incorporating responsible computing in our courses, we have also created grading rubric for ethical/responsible computing related assignments.
We have currently targeted a mix of courses: three required courses (spanning the first year, sophomore and senior year) for the BS Computer Science degree, one popular elective and created one new elective. Here is a quick overview of the courses that we targeted:
Interested in using any of the above material in your class?
Next, we provide bit more details on the ethical and/or responsible computing assignments that we created as part of our existing courses:
The first year seminar is composed of six two-weeks modules (consisting of four lectures and two in-class activities) and we picked two modules in the first year seminar.
It is easy to take for granted how much computers are part of our life. This module explores how we got here and what computing means to modern day life, all with a connection to societal issues and ethics.
This week the exploration is historical, from the beginning though modern times.
Here are all the relevant material for week 1 of the module:
This week looks at the societal issues with computing and modern trends in how computers are part of life. Here are all the relevant material for week 2 of the module:
The Societal Computing module of CSE199 introduces students to the idea of Societal Computing, which we define as computing for, in, and ideally, with, society. (We note that this is our own definition of societal computing, but it is not our term. The concept originates in various places, including as the namesake of Carnegie Mellon’s Societal Computing Ph.D. program.)
In the first week, we cover the idea of societal computing and how it can be used as a critical lens through which we can understand today’s applications of machine learning technologies.
Here are all the relevant material for week 1 of the module:
In the second week, we turn to social media, taking another critical lens as we chart the history of social media, its benefits (e.g. as a space for networked counterpublics) and its many, many failings.
Here are all the relevant material for week 2 of the module:
This module was created using Dalia Muller’s Impossible project framework. In CSE 199, students are asked to spend two weeks of their semester imagining what it would take to build a world in which computing could become anti-racist. Starting with the specific case of the use of predictive policing algorithms, they proposed computational and non-computational solutions to the problems exacerbated by technology in society.
For more details, please see the CSE 199 Impossible project webpage. Below is a ~15 minute video that gives an overview of the module (as well as its development):
Here is a shorter version (~5 minutes) of the above video in case you are short on time:
The goal of this project was to show how simple anonymization trick of dropping sensitive information from a data set and releasing it does not guarantee any privacy if e.g. the “anonymized” dataset could be “joined” with another publicly available dataset. E.g. Latanya Sweeney used this idea to figure out the medical details of the MA governor. As another example, In 2007, Netflix held a competition to improve their recommendation algorithms. This required releasing a large amount of anonymous data from their customers’ movie ratings and viewing histories. By combining the Netflix data set with the Internet movie database (IMDb) site data, researchers found a way to link the identity of a Netflix user to a user’s IMDb profile based on the select reviews that they published.
A two-part homework module was co-developed at UB and Alfred University and piloted in Fall 2021 and Spring 2022 respectively.
Students completed a project (in Scala) where they de-anonymized obscured (mock) private information using (mock) publicly available datasets. The project was followed by a reflection piece that encouraged students to explore publicly available government data for potential de-anonymization risks.
Here are the resources for the reflections piece:
This project was in python and required students to
There are multiple activities/assignments in CSE 331 that all have the common thread of accessing high speed Internet. The main reason for this choice was because this is an issue all of our students could appreciate at a personal level. COVID-19 hit after we implemented/ran these activities in Fall 2019. In future iterations we plan to incorporate increased awareness of the structural inequalities when it comes to access to high speed Internet (see e.g. this Netgain Partnership webinar on Broadband Access in the US) that has been brought to the fore by the pandemic.
Below we provide an overview of four activities/assignments that we piloted in Fall 2019.
The introductory lecture asks students to think about how they would improve broadband access in Cattaraugus county (which is one county over from Erie, which is where Buffalo resides). Students are asked to talk in small groups in the lecture to think about both technical as well as societal issues as they think about how to solve the whole problem.
This in-class activity on improving Internet access in Cattaraugus county was an activity that was used in CSE 331 before Fall 2019 though the focus was mostly on the technical aspects. Starting in Fall 2019, students were explicitly asked to think about societal issues as well.
This related talk provides more details on this in-class activity including some of the in-class responses.
Greedy algorithms is perhaps the basic algorithmic technique. CSE 331 covers greedy algorithms for bit over 2 week period. One of the homework problems for greedy algorithms (which is a classical covering an interval with minimum number of fixed width intervals that can “slide” as needed) asks the students to provide high speed Internet coverage on a straight line using wireless network towers. The problem is connected to the in-class discussion on access to high speed Internet from the first lecture.
Here are the details on the actual problem statement (with some additional notes).
CSE 331 spends two weeks on dynamic programming. One of the homework problems is for students to design a dynamic programming algorithm for a problem that generalizes the weighted interval scheduling problem. The setup asks the students to design an algorithm that schedules request to access a computer with high speed Internet access in a way that serves the diversity of users in the community.
Here are the details on the actual problem statement (with some additional notes).
This project was created from scratch. One constraint we put in early was to make sure that the underlying technical concept that was covered early in the course. Given the current course structure the best option seemed to pick something related to the shortest path problem. On the other side, acknowledging that our student population is skewed towards students from WNY and New Yor State (NYS) in general), we wanted to pick a scenario that was local. During the discussion stage someone suggested we look into the NYS Attorney General’s case against Spectrum that alleged serious unethical behavior of Spectrum towards their customer. Given the ubiquity of the Internet in what we do as computing professionals this seemed like a scenario that all of our students would relate to. The choice was finalized when we realized that we could create a programming project related to routing of packets in this context, which would tie directly to the shortest path problem that we had covered in class (and on which students have a programming assignment early in the semester). The programming project has multiple parts where most of them corresponded to potentially unethical ways of routing packets as detailed in NYS AG’s complaint. The final part asked the students to route packets under a fairness constraint.
Here are the details on the actual coding project (with some code templates).
Ethics is covered in bits and pieces throughout the term. In addition, there is an entire week specifically dedicated to ethics, which has two components: an in-class activity and the capstone project itself.
Students review a made-up dossier of government data to identify if an individual should be added to the government’s No-Fly list. Students must then discuss biases in the data provided and, finally, how they would respond if their job assigned them to the team automating this analysis.
This activity is organized as follows. It (usually) begins with the in-class activity to get them thinking about how these decisions will matter to them. This is then followed up with a brief lecture, presentation of the grading rubric, and then a discussion of the rubric and lecture.
Finally, we have another activity as part of a lecture asking students to discuss the ethics of their project. Students answers to this latter activity are evaluated using the rubric. The results of this latter activity (evaluated using the rubric) are also reported as part of our ABET evaluations.
The machine learning course introduced a new coding group project exploring bias in data driven criminal risk assessment. We used the COMPAS recidivism dataset and data driven criminal risk assessment. There already had been media coverage of this topic and so we figured we could use the coverage to introduce the topic to the students.
The students work on a class assignment (in groups of three) in which they take one of two roles. In either role, each group takes the role of machine learning engineers who have been tasked with designing a new system for use in U.S. court systems. They are given three different ML models and must apply five postprocessing techniques onto each of them. Finally, each group must identify a single model and technique to submit, which is then measured against the rest of the class for a chance at extra credit.
In this role, the students approach this assignment as employees of a large publicly traded corporation. They have a fiduciary duty to their shareholders to deliver strong returns, which can only be done by consistently winning development contracts. To this end, financial considerations are very important, as the engineers want their contracts to be more appealing than your competitors. Additionally, their life relies on the paycheck provided by this job, and making decisions that lose the company money could very quickly get then fired. However, since the corporation is publicly traded, they are still subject to public scrutiny, and thus cannot completely ignore social concerns.
Here is a link to the project description for this role.
In this role, the students approach this assignment as volunteers of a humanitarian NGO (non-governmental organization). The NGO is a non-profit, and thus the volunteers do not receive a paycheck and are not reliant on this work for money to survive. Each group’s mission revolves around advocating for the fair treatment of individuals within the U.S. criminal justice system. To this end, societal considerations and positive reform are the main concerns. However, it will be difficult to get various local and state governments to adopt the software if the financial concerns are completely disregarded.
Here is a link to the project description for this role.
We created a rubric to assess ethics related assignments, which was then used in both CSE 331 and CSE 442. Somewhat interestingly, when we started assessing student work the rubric was not effective. The problem was not with the rubric or student work, but that the assignments discussed ethical issues but did not require students perform any analysis. We first used this rubric while rewriting existing assignments and creating ones to make certain they require students respond to the ethical issues they raised. Once this was complete, in CSE 442, we then presented the rubric to teach students the different criteria for them to consider when evaluating the ethics of a situation and, finally, to evaluate their responses.
Here is a link to the ethics grading rubric.
We encourage you to use the material posted above in your classes. However, we request that you let us know if you have used any of the material above so that we can keep track (in the list below) of schools that have used this material.