Side note1: Sort of out of shape now. BUT! I am trying to keep doing exercise and living in a healthier manner now! Hope to find better me soon! Stay tuned!
Side note2: This is Yin-Yang and I'm trying to integrate the concept of yin and yang into my daily life.
Hi, I'm Zeyi Liao.
I'm a second-year PhD student fortunately advised by Prof. Huan Sun at The Ohio State University. Before that, I worked with Prof. Xiang Ren during my Master study at USC and obtained my Bachelor's degree from BJTU.
I'm broadly interested in AI with a focus on NLP and multimodal AI. My career goal is to build helpful yet robust and trustworthy AI system to make life easier.
My current research preference (subject to change):
- Critically examining and challenging so-called "true claims" in AI through retrospective analysis
- Delving beyond superficial observations to uncover the underlying mechanisms and principles
Fun Fact (Aug 2024): I just aware that my first paper AmpleGCG has aligned my above two preferences. So lucky am I.
What's New
Sep 2024
Feel excited to announce that our investigation into long-tail knowledge is accepted at EMNLP 2024!
Sep 2024
Our new preprint: Environmental Injection Attacks (EIA) at here. This work explores the privacy risks assoticated with the web agent. EIA is one form of indirect prompt injection but specifically targets the environment where state-changing actions happen. We design two injection strategies tailored to the web environments and explore different positions within the webpage to identify the vulnerable regions. More importantly, we provide implications about the levels of the human supervision to banlance the trade-off between autonomy and security, and discuss different defensive approaches, both for pre- and post-deployment stage of the website, with their limitations. Feel free to check the X post here as well.
Aug 2024
Release the raw datasets of AmpleGCG-plus, containing millions of optimized suffixes with their corresponding evaluation results. Check out more details in here. Should be very useful for your if you'd like to build sth upon the GCG.
Aug 2024
July 2024
Thrilled to announce that AmpleGCG has been accepted at COLM 2024.
June 2024
Don't waste your demonstration data and utilize them for joint preference learning. Check out our preprint "Joint Demonstration and Preference Learning Improves Policy Alignment with Human Feedback" here. This is my first time delving into the field of RL and I believe I will have more chances to dig deeper into it in the future.
April 2024
Very proud to have my first author paper AmpleGCG: Learning a Universal and Transferable Generative Model of Adversarial Suffixes for Jailbreaking Both Open and Closed LLMs. Really learn a lot from the journey and can not do it without the help from my advisor. Check out the Twitter post at here.
March 2024
RAG is popular technique to reduce hallucination and provide up-to-date knowledge to static parameteric memory. Wonder how hard is it for LLM to attribute the generation back to the provided reference? Check out the AttributionBench here.
Jan 2024
Agents are growing like viruses. But how can we ensure their safety? Check out our paper "A Trembling House of Cards? Mapping Adversarial Attacks against Language Agents", investigating the security of agents by mapping adversarial attacks from LLMs to Agents.
Nov 2023
Finally, our paper "In Search of the Long-Tail: Systematic Generation of Long-Tail Knowledge via Logical Rule Guided Search" is arxived. One take I have is that: always play around with the long-tail data to examnie the true capability of models. I feel incredibly fortunate to have the opportunity to collaborate with renowned advisors like Yejin Choi and Xiang Ren, especially considering I've only been studying NLP for less than one year.
Aug 2023
Start my Phd journey @ OSU, guided by Prof. Huan Sun. IDK what will happen and I am bit nervous, but also excited, honestly as I have little to no experience in the NLP field and computer science. But who knows, right? Let's see
Selected Publications
EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakage
Zeyi Liao*, Lingbo Mo*, Chejian Xu, Mintong Kang, Jiawei Zhang, Chaowei Xiao, Yuan Tian, Bo Li, Huan Sun
Preprint on Arxiv, 2024 PDF
Joint Demonstration and Preference Learning Improves Policy Alignment with Human Feedback
Chenliang Li, Siliang Zeng*, Zeyi Liao*, Jiaxiang Li, Dongyeop Kang, Alfredo Garcia, Mingyi Hong
Preprint on Arxiv, 2024 PDF
Amplegcg: Learning a universal and transferable generative model of adversarial suffixes for jailbreaking both open and closed llms
Zeyi Liao, Huan Sun
Conference on Language Modeling (COLM), 2024 PDF, X on AmpleGCG-plus
AttributionBench: How Hard is Automatic Attribution Evaluation?
Yifei Li, Xiang Yue, Zeyi Liao, Huan Sun
Annual Conference of the Association for Computational Linguistics (Findings of ACL), 2024 PDF
A Trembling House of Cards? Mapping Adversarial Attacks against Language Agents
Lingbo Mo, Zeyi Liao, Boyuan Zheng, Yu Su, Chaowei Xiao, Huan Sun
Preprint on Arxiv, 2024 PDF
In Search of the Long-Tail: Systematic Generation of Long-Tail Knowledge via Logical Rule Guided Search
Huihan Li, Zeyi Liao*, Yuting Ning*, Siyuan Wang, Xiang Lorraine Li, Ximing Lu, Faeze Brahman, Wenting Zhao, Yejin Choi, Xiang Ren
Empirical Methods in Natural Language Processing (EMNLP), 2024 PDF
Chatcounselor: A large language models for mental health support
June M Liu, Donghao Li, He Cao, Tianhe Ren, Zeyi Liao, Jiamin Wu
ACM International Conference on Information and Knowledge Management (CIKM), 2023 PDF
RobustLR: A diagnostic benchmark for evaluating logical robustness of deductive reasoners
Soumya Sanyal, Zeyi Liao, Xiang Ren
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2022 PDF
Contact