About Me
I am a PhD candidate in Computer Science at the Courant Institute of Mathematical Sciences, New York University, and hold an affiliation with NYU Shanghai. Currently, I am also a visiting scholar in the Machine Learning Department at MBZUAI. My research is conducted under the supervision of Prof. Gus Xia in Music X Lab, where I explore the intersection of music and machine learning.
In 2019, I earned my undergraduate degree in Mathematics from Fudan University. Beyond my academic pursuits, I am a passionate conductor, pianist, and Erhu (a traditional Chinese string instrument) player. I have previously served as the conductor of the NYU Shanghai Jazz Ensemble and as the director of the Fudan Musical Club.
Research Interest
The primary goal of my research is to make intelligent systems more human-like, and I find music to be one of the rich domains to explore this question in depth. My research is highly cross-disciplinary, combining elements of Computer music, Generative AI, Representation Learning, Interpretability, Style Transfer, and Human-Computer Interaction.
Particularly, I design generative models that learn to utilize the hierarchical and nuanced music concepts in music creation, and I develop methods that enable machines to emerge human-interpretable symbolic languages and concepts. I also leverage these methods to design controllable and interactive applications for music creation and education. I hope that through research, we understand ourselves better and we create a more intuitive human-machine co-creation experience.
Here is a brief overview of my research, organized by topic:
Hierarchical Music Generation & Arrangement
- Whole-song generation via compositional hierarchy
- [WholeSongGen]
- Modeling long-term context dependency
- [AccomontageIII][Mixed-level Inpainting]
- Dataset
- [POP909]
Representation Learning
- Representation Disentanglement: monophonic & polyphonic
- [EC2-VAE][Poly-Dis]
- Cross-modality representation: audio & text
- [Audio2Midi][BUTTER]
Unsupervised Concept Emergence
- Learning content & style via variability constraints
- [V3]
- Unsupervised modeling of music structure
- [PianoTree-VAE][MuseBERT]
