Senior Backend Engineer- Evaluations

MagicSchool AI • Full-time • Remote (Remote from USA) • $160,000 - $205,000 / year • 1m ago

Fast Facts

MagicSchool is seeking a Senior Backend Engineer to enhance their Generative AI Evaluations capabilities, focusing on application and data engineering in a remote setting.

Responsibilities: Build tools and processes for evaluating AI outputs, ensure product safety, collaborate with cross-functional teams, and manage data modeling in various storage solutions.

Skills: Expertise in Python and SQL (PostgreSQL), experience with Docker, data modeling, and strong communication skills essential.

Qualifications: 7+ years of engineering experience, including senior roles, with a background in LLMs and Generative AI systems preferred.

Location: USA

Compensation: $160000 - $205000 / Annually

WHO WE ARE: MagicSchool is the premier generative AI platform for teachers. We're just over 2 years old, and more than 6 million teachers from all over the world have joined our platform. Join a top team at a fast growing company that is working towards real social impact. Make an account and try us out at our website and connect with our passionate community on our Wall of Love.

As a Senior Backend Engineer- Evaluations, you will work with the Trust, Safety and Quality team as a strong individual contributor to expand our Generative AI Evaluations capabilities. This work involves both application and data engineering. You will feel a sense of ownership over the space far beyond just taking tickets. You will obsess over safety, quality, speed and user impact. You will know when it's time to build versus time to buy and integrate. You will be passionate about building resilient scalable systems to ensure our users have the best experience possible.

What You Will Do:

Ensure we're building the safest product MagicSchool can build.
Build tools and processes for evaluating MagicSchool output at scale
Stay up to date on the latest tools and literature
Passionate about solving education problems with technology
Work closely with Evaluations product manager, team lead, data scientist and engineer and deliver high quality features
Design, architect, and write high quality code to expand our Evaluations framework's capabilities
Debug complex code and applications in a cloud environment
Build software that is easy for others to understand and easy to maintain
Data modeling in a relational database, data warehouse, and object stores.
Assess and help the team choose appropriate third party solutions to problems when building doesn't make sense
Work cross-functionally to integrate MagicSchool's evaluations platform with the larger application

Qualifications/Competencies/Skills:

Working knowledge of SQL (PostgreSQL) and database design / data modeling
Expert knowledge of Python
Modern Python features like typehints and asyncio
Python Multiprocessing to support evaluation jobs in the 10s to 100s of thousands per run.
Familiarity with Pydantic
Strong communication skills: team-first mindset, highly collaborative, can articulate decisions within team's context
Experience working with and deploying docker applications
Builds relationships easily: emotionally intelligent, communication, warm
Gets a lot done: Works hard, resourceful, do whatever it takes
Adaptable: Smart, learns fast, curious
Nice to Have:
Experience with evaluating generative AI systems
TypeScript and React (useful for internal tooling and cross-team contributions)
AWS experience (bedrock, E(C|K)S)

Experience:

7+ years as an engineer, 2 of which were at senior level or above
Start up experience
Experience with LLMs and Generative AI APIs and systems

Why Join Us?

Work on cutting-edge AI technology that directly impacts educators and students.
Join a mission-driven team passionate about making education more efficient and equitable.
Flexibility of working from home, while fostering a unique culture built on relationships, trust, communication, and collaboration with our team - no matter where they live.
Unlimited time off to empower our employees to manage their work-life balance. We work hard for our teachers and users, and encourage our employees to rest and take the time they need.
Choice of employer-paid health insurance plans so that you can take care of yourself and your family. Dental and vision are also offered at very low premiums.
Every employee is offered generous stock options, vested over 4 years.
Plus a 401k match & monthly wellness stipend

Our Values:

Educators are Magic: Educators are the most important ingredient in the educational process - they are the magic, not the AI. Trust them, empower them, and put them at the center of leading change in service of students and families.
Joy and Magic: Bring joy and magic into every learning experience - push the boundaries of what’s possible with AI.
Community: Foster community that supports one another during a time of rapid technological change. Listen to them and serve their needs.
Innovation: The education system is outdated and in need of innovation and change - AI is an opportunity to bring equity, access, and serve the individual needs of students better than we ever have before.
Responsibility: Put responsibility and safety at the forefront of the technological change that AI is bringing to education.
Diversity: Diversity of thought, perspectives, and backgrounds helps us serve the wide audience of educators and students around the world.
Excellence: Educators and students deserve the best - and we strive for the highest quality in everything we do.