Posts

Each year, Cadence, a computational software company focusing on tools for electronic design automation, awards its Women in Technology Scholarship to support and celebrate young women who are starting their careers. Recently, Brown CS student Sreshtaa Rajesh was declared one of the winners, earning a $5,000 stipend. "Your impressive academic achievements, professor recommendations, and drive to shape the future of technology set you apart from the many talented women we considered," writes Academic Network Program Manager Mallory Clemons of Cadence. "We are excited for what the future holds for you and the impact you will make in technology."

Sreshtaa's letters of recommendation were written by Brown CS Professors R. Iris Bahar, who worked with her on a summer research project, and Shriram Krishnamurthi, her CS advisor, with whom she's taken courses and for whom she served as Head Teaching Assistant. Iris explains that Sreshtaa's work with her was broadly applicable to artificial intelligence planning in domains where there is uncertainty, such as partial occlusion or inaccurate sensor readings. These types of problems can be represented by frameworks known as Partially‐Observable Markov Decision Processes (POMDPs). A POMDP features a set of possible “states” that the agent could be in, as well as a probability distribution over the set, known as the belief state. Sampling from the belief state informs the agent what state it is most likely in, given all the previous observations it has gotten from the environment. Each time the agent takes an action, it receives a new observation which it then uses to update this belief state and inform future actions.

There are several algorithms that exist to solve POMDPs: one such is Partially‐Observable Monte Carlo Planning (POMCP). At each step in the process, the algorithm runs many random look‐ahead simulations to determine the best action to take next. The challenge for using this algorithm is that in order to achieve a “high reward” result, a large number of simulations are required, making the process extremely computationally expensive for large domain real‐world problems. Sreshtaa worked closely with one of Iris's PhD students to parallelize the most expensive part of POMCP: building the look‐ahead search tree and running the simulations.

"Oftentimes," Iris writes in her recommendation letter, "when I assign undergraduate students to work with my graduate students, they just follow specific directions given to them. In Sreshtaa’s case, she was very proactive in offering suggestions, testing out new ideas, and helping implement and debug the parallel algorithm. Specifically, she developed different reward functions for selecting the best action, translated naïve implementations of the code to one using pthreads, explored the use of hashmaps to improve code efficiency, and wrote a plotting script to parse the output of our algorithm. In the end, with the parallelized version, we were able to cut down the runtime of the original POMCP algorithm by almost 50%."

For more information, click the link that follows to contact Brown CS Communication and Outreach Specialist Jesse C. Polhemus.