Abstract: We consider stochastic sequential learning problems where the learner can observe the average reward of several actions. Such a setting is interesting in many applications involving monitoring and surveillance, where the set of the actions to observe represent some (geographical) area. The importance of this setting is that in these applications, it is actually cheaper to observe average reward of a group of actions rather than the reward of a single action. We show that when the reward is smooth over a given graph representing the neighboring actions, we can maximize the cumulative reward of learning while minimizing the sensing cost. We propose CheapUCB, an algorithm that matches the regret guarantees of the known algorithms for this setting and at the same time guarantees a linear cost again over them.
Bio: Manjesh Kumar Hanawal received Ph.D. degree at INRIA, Sophia Antipolis, France, and the University d’Avignon, Avignon, France in 2013. Before this he received B.E. degree in electronics and communication from the National Institute of Technology, Bhopal, India , and the M.Sc. degree in electrical communication engineering from the Indian Institute of Science, Bangalore, India. Currently, he is a Postdoctoral associate at Boston University, USA. His bachelor's studies was sponsored by Airtel India selecting him as a Bharti scholar. His masters’ thesis was awarded best M.Sc. thesis medal (Prof. F. M. Mowdawalla Medal), and he is recipient of INSPIRE faculty fellowship from DST, Government of India.
From 2004 to 2007, he was with the Centre for Artificial Intelligence and Robotics (CAIR), DRDO, Bangalore, as Scientist-B.
