Viviane Clay
Current Work
[Doctoral Dissertation]
The Role of Task and Environment in Biologically Inspired Artificial Intelligence: Learning as an Active, Sensorimotor Process
I wrote my doctoral dissertation on the intersection of machine intelligence and biological intelligence research. I focus on commonalities and differences between the two and how making machine intelligence work more along principles of human intelligence can lead to improved abilities of these systems.
Learning Semantically Meaningful Representations Through Embodiment
How do humans acquire a meaningful understanding of the world with little to no supervision or semantic labels provided by the environment? I investigate embodiment with a closed loop between action and perception as one key component in this process.
I try to apply the idea of embodied learning by training reinforcement learning agents in 3D virtual environments with high dimensional visual observations. This is done with little to no external supervision and learning is sometimes aided by curiosity. I investigate the kind of representations of the sensory input that are learned in the embodied agents, their sparseness, similarity to representations found in animals, their semantic content, robustness, generalizability as well as their similarities to representations found in supervised neural networks.
I further hypothesize that several of the common shortcomings of conventional ANNs such as their vulnerability to adversarial attacks and over-reliance on low-level features originate from the unnatural way in which they are trained. My goal is to show that embodied and curiosity driven exploration of the world leads to more robust and disentangled representations of objects and can even make close to zero-shot object labeling possible.
Deep Reinforcement Learning Workshop
I organized a deep reinforcement learning workshop in January 2021 where we planned a 1 1/2 day DRL crash course followed by some really cool invited speakers from the field and two panel discussions with them. I gave a crash course on reinforcement learning as well as two extra lectures on RL in psychology and RL in neuroscience.
The live talks and panel discussions as well as my course content are availabe on youtube:
https://www.youtube.com/playlist?list=PLlVUF9l2EowbdI05VOaMp73p-TtZjQW7A
Past Projects
A Quantitative Analysis of Artistic Styles
We use artificial neural networks (CycleGANs) to transform photographs into paintings in different artistic styles. Then we use these fake paintings in psychological experiments to investigate whether there are measurable behavioral differences in viewing imagess from different artistic styles. We then create hirarchical clusterings of artistic styles and compare them to theories of art historians.
Clay*, V., Schrumpf*, J., Tessenow*, Y., Leder, H., Ansorge, U., & König, P. (2020). A quantitative analysis of the taxonomy of artistic styles. Journal of Eye Movement Research, 13(2). https://doi.org/10.16910/jemr.13.2.5
[Hackathon] Predicting the Spread of Disease
During the course of a 1 week hackathon organized by the Robert Koch Institute and the University of Osnabrück we developed a concept for using machine learning to make predictions about the spread of diseases. We also developed an interactive interface to view disease spread in Germany over time.
With thins project we won the first price at the hackathon and a trip to the Robert Koch Institute in Berlin.
Eye Tracking in Virtual Reality
We assesed methods and developed protocols and data processing pipelines for measuring eye movements in virtual reality. During my bachelor I build a large-scale virtual city and set up eye tracking and behavioral measuring of subject in VR during navigation tasks. Over my time as VR lab manager and afterwards several hundrend participants have explored the city and performed navigation tasks.
Clay, V., König, P., & König, S. (2019). Eye tracking in virtual reality. Journal of Eye Movement Research, 12(1). https://doi.org/10.16910/jemr.12.1.3
Embodied Spatial Knowledge Aquisition
Using the large-scale virtual city together with the eye tracking in VR setup described above, we investigate spatial knowledge aquisition in humans. As a baseline condition we look at spatial knowledge aquisition with the help of an interactive map. This is then compared to an embodied condation, namely exploring a virtual city with the use of VR.
König, S. U., Clay, V., Nolte, D., Duesberg, L., Kuske, N., & König, P. (2019). Learning of Spatial Properties of a Large-Scale Virtual City with an Interactive Map. Frontiers in Human Neuroscience. Retrieved from https://www.frontiersin.org/article/10.3389/fnhum.2019.00240
König, Sabine U., Ashima Keshava, Viviane Clay, Kirsten Rittershofer, Nicolas Kuske, and Peter König (2021). “Embodied Spatial Knowl- edge Acquisition in Immersive Virtual Reality: Comparison to Map Exploration.” In: Frontiers in Virtual Reality 2, p. 4. issn: 2673- 4192. doi: 10.3389/frvir.2021.625548.
Progressively Growing Neural Networks
I investigate the ability of neural networks to grow progressively when new data becomes available to be more adaptable in real life applications. I devise a simple way to add new classes into a data set without having to retrain the entire network and how to avoid catastrophic forgetting. Additionally I look at the effect of learning to recognize different classes of objects one after another instead of all at the same time and the advantages of curriculum learning.
[Master Thesis] Kakerbeck, V. (2018). Progressively Growing Neural Networks for Scene Graph Generation from Images (Master thesis, University of Osnabrück, Germany). https://doi.org/10.13140/RG.2.2.26794.44488
For this work I won the AI Talents Price 2019 and an award for extraordinary master thesis 2019 (awarded by ROSEN Gruppe).
Health Predictions
Within a study project during my master I used artificial neural networks to make predictions based on a big data set of hospital patient data. For example the symptoms displayed at hospital admittion can be predictive of the diagnosis, length of stay and treatment.
