News
- 02/2024: Two papers on biology vision foundation model (oral) and web-navigation (poster) have been accepted to CVPR 2024!
- 02/2024: I will be interning at Nvidia Learning and Perception Research Group this summer. Catch me in Seattle!
- 07/2023: Our paper on using large language models for vison-and-language navigation accepted to ICCV 2023. Check the paper here.
- 03/2023: Our SalsaBot work has been accepted to the Embodied AI Workshop at CVPR 2023! Check the (short version) paper here.
- 03/2023: I will be interning at Adobe Research this summer. Catch me in San Jose!
- 03/2022: Our paper on long-horizon vison-and-language navigation accepted to CVPR 2022. Check the paper here.
- 12/2021: I will lead the OSU team to participate in the Alexa Simbot challenge! Check us out here.
Go Bucks!
|
Research
Currently my interests are embodied AI and multimodal foundation models. I am also interested in AI for science and multimodal knowledge base.
|
BioCLIP: A Vision Foundation Model for the Tree of Life
Samuel Stevens, Jiaman Wu, Matthew J Thompson, Elizabeth G Campolongo, Chan Hee Song, David Edward Carlyn, Li Dong, Wasila M Dahdul, Charles Stewart, Tanya Berger-Wolf, Wei-Lun Chao, Yu Su
[preprint]
[website] CVPR 2024 (Oral)
|
Dual-View Visual Contextualization for Web Navigation
Jihyung Kil, Chan Hee Song, Boyuan Zheng, Xiang Deng, Yu Su, Wei-Lun Chao
[preprint] CVPR 2024
|
LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Models
Chan Hee Song, Jiaman Wu, Clayton Washington, Brian M. Sadler, Wei-Lun Chao, Yu Su
[paper] ICCV 2023
[short version] Embodied AI Workshop at CVPR 2023
[website]
|
SalsaBot: Towards a Robust and Generalizable Embodied Agent
Chan Hee Song, Jiaman Wu, Ju-Seung Byeon, Zexin Xu, Vardaan Pahuja, Goonmeet Bajaj, Samuel Stevens, Ziru Chen, Yu Su
[short version] Embodied AI Workshop at CVPR 2023
[long version] Alexa Prize SimBot Challenge Proceedings 2023
|
One Step at a Time: Long-Horizon Vision-and-Language Navigation with Milestones
Chan Hee Song, Jihyung Kil, Tai-Yu Pan, Brian M. Sadler, Wei-Lun Chao, Yu Su
[paper] CVPR 2022
|
Adobe Research
Research Scientist Intern
05/2023 -> 08/2023
- I worked on an video editing assistant, with emphasize on open domain code generation problem with large language models. Worked with Gang Wu, Haoliang Wang, and Uttaran Bhattacharya.
|
Human Language Technology Center For Excellence, Johns Hopkins University
Visiting NLP/ML Researcher, SCALE program
05/2019 -> 08/2019
- I implemented various state-of-the-art neural architectures and adapted to named entity recognition task.
Introduced a lexical feature generated from gazetteers to the named entity recognition (NER) system.
As an independent project, developed a novel named entity recognition system achieving state-of-the-art F1 score on Chinese datasets using glyph features generated from convolutional neural network.
|
Kyndi, San Mateo CA
Software Engineering Intern
01/2019 -> 05/2019
- I migrated in-memory graph to external database by building a GraphQL server to interact with the external database using Apollo and Typescript.
Also improved accuracy and representation of the knowledge graph by writing unique graph operations on a large knowledge graph.
|
|
Using Chinese Glyphs for Named Entity Recognition
Chan Hee Song, Arijit Sehanobish
09/2019
extended abstract, full paper, AAAI-20 student abstract
Used Chinese glyph information to augment neural named entity system.
Introduced a convolutional neural network (CNN) architecture that achieves a state-of-art performance on Chinese Weibo NER dataset and sets a new baseline F1 score on Chinese OntoNotes v5.0 dataset.
Easy to implement, train and uses less data. Robust to non-Chinese characters in the dataset.
|
|
Gazetteer Generation for Neural Named Entity Recognition
Chan Hee Song, Dawn Lawrie, Tim Finin, Jim Mayfield
09/2019
paper, FLAIRS-33
Introduced a simple method to generate gazetteers from the Web.
Generated gazetteer is used as a feature to augment neural named entity recognition system, showing improvement over baseline.
Developed a method to create new training data using gazetteer entity replacement.
Presented new Russian NER corpus gathered from Reddit.
|
|
Sentence Representation for Neural Machine Translation Systems
Chan Hee Song, David Chiang
06/2018 -> 05/2019
Independent undergraduate researcher under supervision of professor David Chiang.
I investigated the impact of sentence representation on neural machine translation to develop an optimal encoding mechanism that has a better trade-off between time and accuracy.
I am using C, Python and PyTorch library to build encoder/decoder, modify neural machine translation system, and implement new equations.
|
|
Verba Volant, Scripta Manent: Automatic Transcription of Medieval Latin Manuscripts
Undergraduate Research Assistant
01/2018 -> 05/2018
poster
Research assistant to the project which aims to find a novel method to process Vatican Secret Archives to transfer early Medieval Latin handwritten papers into a computer readable format.
I analyzed various character recognition algorithm and used Python to build an auto analyzer tool for Optical Character Recognition systems
|
|