Principal Engineer, Software - Backend, AI Observability/Evaluations (Chronosphere)

Palo Alto NetworksKorumburraRemoteFull-time2 weeks ago

Palo Alto Networks is hiring a Principal Engineer for its Chronosphere observability platform to build AI Observability and Evaluation capabilities. The role involves designing scalable backend systems, APIs, and data processing for testing, measuring, and benchmarking AI models, agents, and RAG applications. Requires 8+ years of software engineering experience and strong backend language skills.

What you'll do

Build backend capabilities that help developers observe and evaluate AI models, agents, and RAG applications before and during production
Design systems for systematic testing, measurement, and benchmarking of AI agents and applications accuracy, relevance, performance, and safety
Build scalable APIs and data processing systems that contextualize AI telemetry and evaluation results
Help shape the technical roadmap and architecture for AI Observability and Evaluation
Collaborate with product managers, deeply technical teammates, and customers to scope, validate, and deliver integrated features and developer workflows
Improve our own system observability
Improve the performance, reliability, and cost efficiency of the distributed systems powering AI Observability and Evaluation
Mentor engineers and help maintain our engineering standards

What you need

Must-have

GoJavaPythonRustAWSGCPAzureREST APIs

Nice-to-have

LLM powered product features

Soft skills

CollaborationAutonomyJudgementMentoringProduct sense

Principal EngineerBackendAI ObservabilityAI EvaluationsChronosphereCybersecurityRemoteGoPythonLLM

/What you'll do

/What you need

Must-have

Nice-to-have

Soft skills

What you'll do

What you need