Scholar Spotlight: Hassan Shahmohammadi

After obtaining his master's degree in Computer Science in Iran, he commenced his doctoral research journey with IMPRS-IS in January 2020, affiliating with the computer graphics group at the University of Tübingen under the guidance of Professor Hendrik Lensch and Professor Harald Baayen. His research pursuits lie at the intersection of computer vision and NLP, primarily focusing on the visual grounding of language. Where he tries to build multimodal embeddings that are more cognitively plausible and in line with human judgment.

In December of the preceding year, Hassan's recent project, ViPE (Visualize Pretty-much Everything), won the outstanding paper award at the EMNLP conference in Singapore. ViPE is the first language model specifically tailored to support text-to-image models such as DALLE and SDXL for handling figurative language and abstract visualization. His model generates visualizable prompts for any textual input, facilitating the visualization of abstract concepts like freedom and metaphors such as 'bite off more than you can chew'. ViPE is engineered to distill knowledge from ChatGPT through symbolic knowledge distillation and has demonstrated greater robustness than GPT3.5 Turbo (ChatGPT) across various evaluation benchmarks. ViPE presents diverse downstream applications, including music video generation and abstract visualization. Check out the demos of some of these applications:

Music Video Generation: https://github.com/Hazel1994/ViPE-Videos
Document summarization through Images: https://huggingface.co/spaces/fittar/summagary
Prompt Generation: https://huggingface.co/spaces/fittar/ViPE

Beyond his academic pursuits, Hassan derives enjoyment from creating artistic works (https://www.instagram.com/fittar.art/) and actively engages in various sports. He is currently working on his PhD thesis and from the first of May he will join the SONY research team in Stuttgart as a senior research scientist.