Holy Lovenia

I’m pursuing an MPhil at HKUST and working on vision-language & NLP research as a part of Centre for Artificial Intelligence Research (CAiRE).

Previously

Before coming to Hong Kong, I was first introduced to research in the summer of 2018, when I became a research intern in AHC Lab at Nara Institute of Science and Technology (NAIST), Japan, working on brain signal processing research.
The next year, I got my Bachelor’s degree from the undergraduate program of Computer Science at Bandung Institute of Technology (ITB), Indonesia.
Then, from 2019 to 2021, I worked on various projects regarding NLP, speech synthesis, and paralinguistics with Prosa.ai.

Interests

As of now, I’m researching in the field of multimodality (vision-language), multilinguality, and conversational AI.

You can take a look at my resume here.

If you are interested to collaborate or discuss, please feel free to reach out to me.

News

Aug 24, 2023	Oral presentation of our paper “Cross-Lingual Cross-Age Group Adaptation for Low-Resource Elderly Speech Emotion Recognition” at INTERSPEECH 2023!
Jul 19, 2023	Defended my MPhil thesis on “Advancing Vision-Language Understanding of Objects in Multimodal Conversational AI” and got an excellent remark from the committee.

Selected Publications

A multitask, multilingual, multimodal evaluation of ChatGPT on reasoning, hallucination, and interactivity

Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, and others.

arXiv preprint arXiv:2302.04023. 2023.

arXiv Bib

@article{bang2023multitask,
  title = {{A multitask, multilingual, multimodal evaluation of ChatGPT on reasoning, hallucination, and interactivity}},
  author = {Bang, Yejin and Cahyawijaya, Samuel and Lee, Nayeon and Dai, Wenliang and Su, Dan and Wilie, Bryan and Lovenia, Holy and Ji, Ziwei and Yu, Tiezheng and Chung, Willy and others},
  year = {2023},
  journal = {arXiv preprint arXiv:2302.04023},
}

NusaCrowd: Open Source Initiative for Indonesian NLP Resources

Samuel Cahyawijaya, Holy Lovenia, Alham Fikri Aji, Genta Indra Winata, Bryan Wilie, Rahmad Mahendra, Christian Wibisono, Ade Romadhony, Karissa Vincentio, Fajri Koto, and others.

In Findings of the Association for Computational Linguistics: ACL 2023, Toronto, Canada, 9-14 July 2023. 2023.

arXiv Bib PDF Supp Code Poster

@inproceedings{cahyawijaya2022nusacrowd,
  title = {{NusaCrowd: Open Source Initiative for Indonesian NLP Resources}},
  author = {Cahyawijaya, Samuel and Lovenia, Holy and Aji, Alham Fikri and Winata, Genta Indra and Wilie, Bryan and Mahendra, Rahmad and Wibisono, Christian and Romadhony, Ade and Vincentio, Karissa and Koto, Fajri and others},
  year = {2023},
  journal = {arXiv preprint arXiv:2207.10524},
  booktitle = {Findings of the Association for Computational Linguistics: {ACL} 2023, Toronto, Canada, 9-14 July 2023},
  publisher = {Association for Computational Linguistics},
  series = {Findings of {ACL}},
  volume = {{ACL} 2023},
}

Prompting Large Language Models to Generate Code-Mixed Texts: The Case of South East Asian Languages

Zheng-Xin Yong, Ruochen Zhang, Jessica Zosa Forde, Skyler Wang, Samuel Cahyawijaya, Holy Lovenia, Lintang Sutawika, Jan Christian Blaise Cruz, Long Phan, Yin Lin Tan, and others.

arXiv preprint arXiv:2303.13592. 2023.

arXiv Bib

@article{yong2023prompting,
  title = {{Prompting Large Language Models to Generate Code-Mixed Texts: The Case of South East Asian Languages}},
  author = {Yong, Zheng-Xin and Zhang, Ruochen and Forde, Jessica Zosa and Wang, Skyler and Cahyawijaya, Samuel and Lovenia, Holy and Sutawika, Lintang and Cruz, Jan Christian Blaise and Phan, Long and Tan, Yin Lin and others},
  year = {2023},
  journal = {arXiv preprint arXiv:2303.13592},
}

Which One Are You Referring To? Multimodal Object Identification in Situated Dialogue

Holy Lovenia, Samuel Cahyawijaya, and Pascale Fung.

In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop. May 2023.

arXiv Bib PDF Code Poster Slides

@inproceedings{lovenia-etal-2023-one,
  title = {{Which One Are You Referring To? Multimodal Object Identification in Situated Dialogue}},
  author = {Lovenia, Holy and Cahyawijaya, Samuel and Fung, Pascale},
  year = {2023},
  month = may,
  booktitle = {Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop},
  publisher = {Association for Computational Linguistics},
  address = {Dubrovnik, Croatia},
  pages = {61--72},
}

ASCEND: A Spontaneous Chinese-English Dataset for Code-switching in Multi-turn Conversation

Holy Lovenia, Samuel Cahyawijaya, Genta Indra Winata, Peng Xu, Xu Yan, Zihan Liu, Rita Frieske, Tiezheng Yu, Wenliang Dai, Elham J Barezi, and others.

In Proceedings of the 13th Language Resources and Evaluation Conference (LREC). May 2022.

arXiv Bib Supp Code Poster Slides

@inproceedings{lovenia2022ascend,
  title = {{ASCEND: A Spontaneous Chinese-English Dataset for Code-switching in Multi-turn Conversation}},
  author = {Lovenia, Holy and Cahyawijaya, Samuel and Winata, Genta Indra and Xu, Peng and Yan, Xu and Liu, Zihan and Frieske, Rita and Yu, Tiezheng and Dai, Wenliang and Barezi, Elham J and others},
  year = {2022},
  booktitle = {Proceedings of the 13th Language Resources and Evaluation Conference (LREC)},
}

Clozer: Adaptable Data Augmentation for Cloze-style Reading Comprehension

Holy Lovenia, Bryan Wilie, Willy Chung, Zeng Min, Samuel Cahyawijaya, Dan Su, and Pascale Fung.

In Proceedings of the 7th Workshop on Representation Learning for NLP (RepL4NLP). May 2022.

arXiv Bib PDF Poster Slides

@inproceedings{lovenia2022clozer,
  title = {{Clozer: Adaptable Data Augmentation for Cloze-style Reading Comprehension}},
  author = {Lovenia, Holy and Wilie, Bryan and Chung, Willy and Min, Zeng and Cahyawijaya, Samuel and Su, Dan and Fung, Pascale},
  year = {2022},
  booktitle = {Proceedings of the 7th Workshop on Representation Learning for NLP (RepL4NLP)},
  pages = {60--66},
}

Every picture tells a story: Image-grounded controllable stylistic story generation

Holy Lovenia, Bryan Wilie, Romain Barraud, Samuel Cahyawijaya, Willy Chung, and Pascale Fung.

In Proceedings of the 6th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature. Oct 2022.

arXiv Bib PDF Poster Slides

@inproceedings{lovenia-etal-2022-every,
  title = {{Every picture tells a story: Image-grounded controllable stylistic story generation}},
  author = {Lovenia, Holy and Wilie, Bryan and Barraud, Romain and Cahyawijaya, Samuel and Chung, Willy and Fung, Pascale},
  year = {2022},
  month = oct,
  booktitle = {Proceedings of the 6th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature},
  publisher = {International Conference on Computational Linguistics},
  address = {Gyeongju, Republic of Korea},
  pages = {40--52},
  url = {https://aclanthology.org/2022.latechclfl-1.6},
}

Speech artifact removal from EEG recordings of spoken word production with tensor decomposition

Holy Lovenia, Hiroki Tanaka, Sakriani Sakti, Ayu Purwarianti, and Satoshi Nakamura.

In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Oct 2019.

arXiv Bib PDF Poster

@inproceedings{lovenia2019speech,
  title = {{Speech artifact removal from EEG recordings of spoken word production with tensor decomposition}},
  author = {Lovenia, Holy and Tanaka, Hiroki and Sakti, Sakriani and Purwarianti, Ayu and Nakamura, Satoshi},
  year = {2019},
  booktitle = {ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
  pages = {1115--1119},
  organization = {IEEE},
}