Project: Curiosity driven fairness in Reinforcement learning
Description
Reinforcement learning (RL) is a computational approach to automating goal-directed decision making. In this project, we will use the framework of Markov decision processes.
Fairness in reinforcement learning [1] deals with removing bias from the decisions made by the algorithms. Bias or discrimination in this context refers to unfavorable treatment of people due to the membership to certain demographic groups that are distinguished by the protected attributes such as race, gender, age etc.
Curiosity-driven learning has been studied in many different settings [2,3]. In the field of competitive resource allocation, it has been shown that curiosity-driven learning increases equality[4].
Activities
In this project, we will investigate how curiosity drives fairness in learning processes. We will study how to add curiosity-driven exploration in several ways to RL and deep-RL algorithms and examine the outcomes in terms of fairness. Then, we aim to propose a general way to add curiosity-driven exploration to RL algorithms in order to achieve given fairness constraints.
It is possible to tailor the degree to which the project leans towards either theory or experiments to the skills and interests of the participating student.
Requirements
1. Coding skills and a basic understanding of RL/deep RL algorithms.
2. No previous knowledge in fair-machine learning is required but is a plus.
References
[1] Pratik Gajane, Akrati Saxena, Maryam Tavakol, George Fletcher, Mykola Pechenizkiy. Survey on Fair Reinforcement Learning: Theory and Practice. arXiv:2205.10032
[2] Jürgen Schmidhuber. Formal Theory of Creativity, Fun, and Intrinsic Motivation. In IEEE Transactions on Autonomous Mental Development, 2010.
[3] Marc G. Bellemare, Sriram Srinivasan, Georg Ostrovski, Tom Schaul, David Saxton, Remi Munos. Unifying Count-Based Exploration and Intrinsic Motivation. In advances in Neural Information Processing Systems, 2016.
[4] Bernadette Bucher, Siddharth Singh, Clélia de Mutalier, Kostas Daniilidis & Vijay Balasubramanian Curiosity Increases Equality in Competitive Resource Allocation. In ICLR 2020 workshop, Bridging AI and Cognitive Science
Details
- Supervisor
-
Mykola Pechenizkiy
- Secondary supervisor
-
Pratik Gajane
- Interested?
- Get in contact