Senior Software Engineer -Observability, Managed Inputs Full Time

What is The Role

As part of the Observability organization and inside the Streams program, the Managed Inputs team builds high-performing backend ingestion pipelines. We collect telemetry across diverse environments—including Elastic Agents, agentless setups, OpenTelemetry collectors, and cloud sources via Elastic Cloud Forwarders—then process and store this information to power seamless, intuitive user experiences within Kibana.

Elastic's Streams program is the unified platform for getting data into Elasticsearch from edge collection all the way through cloud processing. We accept any type of data generated at the edge (Elastic Agent, Elastic Classic APM Agents, Beats, any OTel SDK…) and we process and route the data to several destinations.

We are building strong backend solutions for ingesting observability data. Our core focus is on building a performant, reliable, scalable, secure and maintainable backend that fulfills product requirements. We take responsibility for what we build, and therefore are also responsible for keeping the system running, and ensuring reliable quality gates and safe deployment mechanisms, as well as assisting in support cases. 

What You Will Be Doing

• Design, implement, deploy and maintain high-performing, scalable features within the different products we own, being the most critical ones Managed Inputs (mOTLP endpoints) and Managed Integrations (Agentless architecture) across the three main cloud providers.
• Take ownership of projects and features, leading them from concept to delivery.
• Provide engineering level support to escalated customer cases, on a rotation basis.
• Provide mentorship and feedback to peers through pull request reviews, ensuring code quality and promoting best practices within the team.
• Monitor alerts, build SLIs and SLOs and ensure system stability.
• Cultivate a positive, inclusive, and collaborative team culture across timezones.
• You will be on call for services we own (current rotation is 12h/day/week every 8-10 weeks)

What You Bring

• Intermediate to Expert Go skills with strong analytical, debugging, and troubleshooting skills.
• Hands-on experience with Kubernetes, IaC (Terraform) and Kafka
• At least Intermediate experience with Observability (tracing, metrics, logs, profiling) 
• Strong understanding of microservice architecture and distributed systems, ability to understand tradeoffs between different solutions and familiarity with several parts and challenges for designing such systems.
• A positive attitude and a curious mindset with commitment to learning and self-enablement
• Ability to conduct root cause analysis and investigating to find the bottom of issues rather than accepting unexpected system behavior
• Ability to work independently in a globally distributed team, while ensuring to collaborate and share knowledge with colleagues.
• Experience with or interest in OpenTelemetry
• You work autonomously and can drive decisions and results in a distributed team by leveraging asynchronous, direct, and transparent communication.
• Demonstrated experience leveraging AI-assisted development tools to accelerate feature development, debug complex systems, and optimize existing codebases.

Bonus Points

• Experience in designing scalable, high-availability services and applications ingesting terabytes per day.
• Experience with operational stability as responder or incident commander
• Experience with Elasticsearch and/or Kibana
• Passion for open-source development and contributions to community projects (doesn’t have to be ours).
• Experience in full remote teams