Infrastructure Engineer

This is a nontrivial challenge 🙂 💪 Let’s go?

Price feeds run 24x7. DeFi applications depend on the accuracy and availability of these feeds; an inaccurate price or offline feed can cause severe financial losses. Each feed in turn depends on many different services, some of which are run by our data providers and some by us. It’s a complex system with many failure modes, but it must always work correctly.

We also run a variety of off-chain services. These services run in a Kubernetes cluster that is managed using Terraform. We also need to ensure these services are running and always healthy.

We’re looking for people to help us operate this system and improve its reliability over time. This job includes providing front-line support for incidents, developing automation to manage our infrastructure, and defining deployment plans for high availability.

Location: Worldwide, Remote

___

About Our Team and Your Role

We’re a well-rounded team: half of us are tech whizzes, while the other half excel in building partnerships with data providers, developers, and the DeFi community. Communication is key to our network-driven approach.

Remote Work: Our team is spread across the globe, from the US and South America to Europe and Asia, with an optional office in Porto, Portugal. Remote work is the norm here.

Language Proficiency: English proficiency is essential as it’s our primary communication language.

Startup Vibe: We thrive in the dynamic DeFi space and love adaptable problem solvers who are eager to meet evolving market needs.

About The Job:

Infrastructure

____________________

We maintain server infrastructure on the cloud and bare metal. Our applications move a lot of data over the network, and that is our focus.

Most of our infrastructure is provisioned using infrastructure-as-code (IaaC) tools such Terraform and Ansible, and orchestrated with Kubernetes. We avoid provisioning resources manually using a web console or typing commands on an SSH session because that results in losing track of what and why resources were created.

Another important aspect of how we build infrastructure is where it happens: we avoid doing tasks from our laptops and we avoid exposing secrets and keys outside of the context where they are used.

Finally, some of our software runs on blockchains which means we do not have full control over its infrastructure, so we work with each blockchain community of nodes to coordinate on-chain operations.

Operation & Monitoring

____________________

We operationalize the software written by the engineering team. Software tends to break when it is changed, and it changes many times over its lifecycle, so we try to make the integration and deployment process as automated as possible.

Integration means taking the source code and producing a runnable artifact (e.g. a binary or container image) while deployment means taking the produced artifact and running it on a production environment.

Our current model for deploying software is keeping a manifest with the ideal state of all workloads, and running tools that read these manifests and reconcile the current state of the workloads. Part of this model is usually referred to as GitOps.

Once the software is deployed, we need to know how well it is running. This work begins in the development phase where engineers instrument their code with logs and metrics, and we connect the logs and metrics to our monitoring stack. Different workloads have different instrumentation needs.

Development

____________________

We build our own tools. We try to use well-established, off-the-shelf tools, but that is not always possible. Our development work covers tooling used by ourselves, by engineering, and by other teams. Here are some examples of use cases for our tooling:

• Track the balance of blockchain addresses and notify when it goes below a threshold

• Observe price feeds and send Slack messages when something interesting happens

• Configure on-chain programs

• Manage development environment

• Serve a real-time dashboard of price feeds and publishers

____________________

Skills we’d love you to have:

• 4+ years of experience working in Engineering, DevOps, or SysAdmin roles

• Understanding of Linux fundamentals and comfortable operating Linux servers

• Experience packaging and deploying workloads in production-grade environments

• Experience programming in high-level languages such as Go, Python, and/or JavaScript

• Understanding of blockchains and smart contracts

Think you have what it takes? We’re looking forward to meeting you! 👋

🫂 At Douro Labs, we value diversity and inclusion in our workplace. We are an equal-opportunity employer committed to preventing discrimination or harassment. We oppose unfair treatment based on protected characteristics such as race, religion, color, national origin, gender, gender expression, sex, sexual orientation, age, marital status, veteran status, disability status, or any other characteristic protected by law. This policy applies to all aspects of employment, including hiring, promotion, termination, leave, and apprenticeship. Our hiring decisions are based on qualifications, merit, and business needs. We strive to create an inclusive and welcoming environment for everyone.