Audio Generalist Evaluator Expert

Description

Design and Optimize Prompts: Create detailed audio prompts with multiple constraints and instructions. Define and Document Evaluation Standards: Establish high-level expectations for correct responses in general audio consumer contexts, and develop comprehensive rubric. Conduct Model Testing and Grading: Run prompts through models and assess preliminary outputs against expectations. Support Benchmarking and Quality Assurance: Collaborate in QA review processes to ensure prompt tasks and rubrics meet rigor, maintaining consistency and reliability before integration into official benchmarks.

Requirements

• BS or BA from a reputable institution completed or in progress
• Strong writing and critical thinking skills.
• Ability to work independently and meet deadlines.
• Significant familiarity with ChatGPT or similar tools for personal decision-making or hobbies / general interests.
• Preferred Qualifications:
• 2+ years of experience in teaching or research.