Research Comparison & General Master’s/Ph.D. Research Report Evaluator (USA/Canada - Remote)
💡 Tip sa Pag-apply: Ang pag-click sa "Mag-apply sa Braintrust nang Libre" ay magdadala sa iyo sa opisyal na site ng Braintrust. Ito ay 100% libre para sa iyo at nakakatulong na suportahan ang aming platform sa pamamagitan ng mga referral bonus.
⚠️ Paalala sa pagsasalin: Ang impormasyong ito ay isinalin gamit ang AI. Kung may hindi malinaw o may pagkakamali, ang English na orihinal ang dapat sundin.
Role Overview
This is a full-time if USA based (8 hours/day - 40 hours/week commitment) or part-time if Canada based(4 hours/day - 20 hours per week commitment) with at least 4 hours per day and a minimum of 20 hours per week, with a 4-hour overlap with PST. Contract duration is set to an initial 1 week.
About Turing:
Based in San Francisco, California, Turing is the world's leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems. Turing supports customers in two ways: first, by accelerating frontier research with high-quality data, advanced training pipelines, plus top AI researchers who specialize in coding, reasoning, STEM, multilinguality, multimodality, and agents; and second, by applying that expertise to help enterprises transform AI from proof of concept into proprietary intelligence with systems that perform reliably, deliver measurable impact, and drive lasting results on the P&L.
Role overview:
- We are conducting an initiative that involves evaluating deep research reports generated by AI systems. Evaluators will read and assess reports across multiple quality dimensions using a structured rubric, providing both numerical ratings and written justifications.
- This is a one-pass evaluation: each report is reviewed by a single general evaluator. The project covers hundreds of reports spanning diverse domains including technology, healthcare, education, science, economics, social sciences, arts, and more.
What does day-to-day look like:
- Read the research query to understand the intent and scope of the assigned report.
- Read and annotate the report using a PDF reader (Adobe Acrobat), highlighting key evidence and flagging issues such as broken links, weak sources, and claim-source mismatches.
- Score the report across 10 dimensions on a 1-5 scale using the provided rubric.
- Write justifications for each rating, referencing specific evidence from the report.
- Submit results via a survey and return the annotated PDF.
Requirements:
- Graduate student (Master's or PhD candidate).
- Professionals with 1-3 years of experience in a relevant or adjacent domain.
- Regularly reads and consumes research reports, white papers, or analytical documents.
- Strong English reading comprehension and written communication skills.
- Based in the USA or Canada.
- Comfortable working with PDF annotation tools (Adobe Acrobat) and online surveys.
- Familiarity with one or more of the report domains: technology, healthcare, education, science, economics, social sciences, or related fields.
- Experience with quality evaluation, annotation, or data labeling tasks.
- Attention to detail and ability to provide clear, evidence-based written feedback
What Turing is NOT seeking from your expertise:
- Confidential or proprietary information from any employer or university.
- Trade secrets or internal company or university data.
- Specific client information or case details.
- Any information that would violate NDAs, employment agreements or other confidentiality obligations.
Mga Alerto sa Trabaho