Open-Source Repository Benchmark Task Author
Job Description
IXO is engaging specialists to evaluate coding benchmarks grounded in substantive open-source repository work. Your contribution is technical work: make the reasoning inspectable, identify substantive errors and provide evidence that supports a reliable assessment.
Work you will do
• Use real repository knowledge to construct development tasks or review code-generation outputs.
• Implement and test solutions in the relevant language, checking integration behavior and the technical depth of the change.
• Document code context, verification and review reasoning so the task can be assessed reproducibly by another engineer.
Required background and routes
• Repository-contributor route: a verifiable GitHub record showing meaningful ownership or substantive contributions, professional engineering depth and multiple languages; superficial edits do not establish this experience.
• Code-generation research route: competitive-programming/problem-analysis expertise, strong algorithms/data structures, at least one of Go, Rust, Java, C++, Python or TypeScript, and open-source or collaborative-project evidence.
Preferred background
• The advanced contributor route favors expertise in two of C#, Go, TypeScript, Rust, Java, C+• or Python, substantial feature/bug/refactor work and competitive-problem analysis.
• Precise validation and documentation, large-codebase improvement and independent delivery. Prior AI experience is optional.
Deliverables
Submit the completed technical artifact or assessment with its supporting evidence, explicit assumptions, reproducible checks where applicable, and concise reasons for each material judgment. Address review findings within the agreed scope.
Location and schedule
Remote assignments are scheduled by agreement, with no guaranteed weekly volume. Availability planning can include 15 hours per week depending on the track. IXO confirms the applicable timing before work.
Pay and working terms
Per completed task; fee agreed before work USD. Payment is for a completed task that meets the agreed acceptance criteria. The fee, task specification and revision requirements are agreed before work; an advertised time estimate does not determine payment. Applying does not guarantee an assignment. Use public, licensed or otherwise authorized material only; do not submit confidential employer information, personal data or restricted research.