by M-A-P
WildSongBench is a benchmark of 192 song-generation prompts, 94 in Chinese and 98 in English, from the M-A-P group. It ships prompts, exact inputs, seeds, reference scores and a small evaluator, but no audio.
DatasetFree
Step 1: Load the test split with load_dataset("m-a-p/WildSongBench", split="test").
Step 2: Read each record's prompt and the style, lyrics and seeds for generation.
Step 3: Generate songs with your model for all 192 prompts, since audio is not included.
Step 4: Install the evaluator from the repository and run the prepare step.
Step 5: Run the score commands and compare against the reference scores in the card.
No. It provides prompts, exact inputs and seeds, reference scores and an evaluator. You generate the audio yourself.
There are 192 prompts: 94 in Chinese and 98 in English.
SongBench, Q3O, MuQ-MuLan, AllMusicCaps and phoneme error rate.
Q3O requires an 80 GB GPU, and the other metrics support 24 GB. The card also lists Linux and a fresh Python environment.