<div class="show-more-less-html__markup show-more-less-html__markup--clamp-after-5 relative overflow-hidden"> <p>At HRvizer Recruitment Agency, we are partnering with a fast-growing tech startup based in Berlin that is transforming the way audits are performed using AI. By automating repetitive, document-heavy processes, the company enables experts to focus on high-value decision-making.</p><p>The business is backed by leading venture capital funds, has secured over €10M in funding, and already has a live product with paying customers. The team values first-principles thinking, speed, trust, and collaboration, working closely together from their Berlin office.</p><p> </p><p><strong>Your role</strong></p><p>This role brings together backend engineering, data engineering, AI infrastructure, and LLM operations, offering the opportunity to build and enhance the systems that power AI agents in production.</p><p> </p><p>Working directly within backend and agent architecture, you'll develop the tools and infrastructure needed to monitor, evaluate, debug, optimize, and continuously improve AI-driven solutions. You'll write production-ready code, design scalable backend systems, and play a key role in increasing the reliability, performance, efficiency, and cost-effectiveness of our LLM-powered agents.</p><p> </p><p><strong>What you’ll do</strong></p><ul><li>Design and build online and offline evaluation frameworks for LLM-powered agents, leveraging golden datasets, ground-truth data, human review workflows, and experimental results to measure and improve performance.</li><li>Develop automated quality assurance pipelines that validate changes to prompts, models, context, and agent logic before deployment to production.</li><li>Analyze large-scale agent execution data to uncover failure patterns, performance regressions, latency bottlenecks, reliability issues, and opportunities for cost optimization.</li><li>Work with analytical databases and columnar storage technologies, such as BigQuery, ClickHouse, or comparable platforms, to process and interpret large datasets.</li><li>Build robust data retention, replay, and analysis capabilities to support long-term monitoring and continuous improvement of production AI agents.</li><li>Develop observability solutions, including dashboards, logging, tracing, experiment tracking, and debugging tools, to ensure visibility into agent performance and system health.</li><li>Contribute to the core backend and AI agent architecture by enhancing existing agents and developing new capabilities as business needs evolve.</li></ul><p> </p><p> </p><p><strong>What we’re looking for</strong></p><ul><li>5+ years of experience as a <strong>Software Engineer</strong> or <strong>Applied Scientist</strong>, with hands-on expertise in information retrieval and large-scale backend systems.</li><li>Strong foundation in <strong>data engineering</strong>, including designing and maintaining scalable data solutions.</li><li>Proficiency in <strong>Python</strong> and backend development, with the ability to build reliable, production-grade applications.</li><li>Advanced SQL skills and experience working with large, complex datasets.</li><li>Hands-on experience deploying, managing, and optimizing cloud-based systems, preferably on <strong>Google Cloud Platform (GCP)</strong>.</li><li>Proven ability to design and implement data pipelines, ETL/ELT processes, event-driven architectures, and production feedback loops.</li><li>Experience working with analytical databases, data warehouses, columnar storage technologies, and high-volume event or telemetry data.</li><li>Solid understanding of distributed systems, including system design, observability, monitoring, logging, debugging, reliability, and operational best practices.</li><li>Ability to navigate complex codebases, quickly understand existing architectures, and contribute effectively to their evolution.</li><li>Strong engineering judgment, with the ability to make architectural decisions, evaluate trade-offs, and build scalable systems that support long-term growth.</li><li>Comfortable working in fast-paced, evolving environments, solving ambiguous problems from first principles, and building infrastructure that powers AI systems in production.</li></ul><p> </p><p> </p><p><strong>Nice to have</strong></p><ul><li>Experience building and scaling infrastructure for <strong>LLM-powered applications</strong> or autonomous AI agents, with a focus on optimizing model performance, context management, reasoning efficiency, and model selection.</li><li>Hands-on experience analyzing and working with production trace data from complex, distributed systems.</li><li>Proven ability to develop internal platforms and tools that improve productivity for engineering, operations, or domain-specific teams.</li><li>Experience implementing and managing workflow orchestration solutions, such as <strong>Temporal</strong> or comparable technologies.</li><li>Exposure to highly regulated or accuracy-critical industries, including finance, audit, compliance, or similar domains.</li><li>Previous experience thriving in an early-stage startup or other fast-paced, high-growth engineering environment, where adaptability and ownership are essential.</li></ul><p> </p><p>What’s in it for you</p><ul><li>High-impact role in a rapidly scaling AI startup </li><li>Opportunity to shape technical direction from an early stage </li><li>Collaborative, mission-driven environment </li><li>Competitive salary: €100,000 – €130,000 + equity </li><li>Learning and development budget (courses, conferences) </li><li>Flexible vacation policy, team events, and modern office in central Berlin </li></ul> </div>