Portugal, Lisboa, LisboaEngineering
Full Time | Portugal
We’re seeking talented, passionate and pragmatic profiles that are obsessed with working with data and getting value out of it.
Daltix provides highly accurate and accessible data to companies active in the dynamic retail industry. We are obsessed about the quality of our data and we thrive on making it work for our customers so that they can make fact-based business decisions. We are proud to have a solid customer base of some of the largest European retailers such as Colruyt Group, GfK and Unilever. In the next couple of years, Daltix is focused on further fast growth which means our talent will have a significant impact on creation, development and execution of our product and brand. Daltix was founded in 2016 and has teams located in Belgium and Portugal.
About the job:
We are looking for a Software Engineer to join our Data Collection team in Lisbon, Portugal, helping us expand and maintain our data collection and web crawling systems.
The Data Collection team lies at the heart of our Engineering organisation and is responsible for scraping and transforming web data on a massive scale. We crawl tens of millions of URLs per day and process terabytes of raw data per month, all the while maintaining extremely high data quality and coverage guarantees.
In this role you will have the opportunity to be in the front-lines of our web collection activities, using Daltix’ s state of the art distributed crawler framework to build new web scrapers, maintain existing ones and develop data transformers for post-processing of raw data.
You’ll be working in a highly dynamic environment with one of the industry’s most experienced web scraping teams; enjoying direct exposure to challenging and interesting engineering problems as we continue on our journey to collect every retail data point on the web.
Your responsibilities in this role:
- Perform technical assessments of new sites we wish to collect (reverse engineering APIs, decompling mobile apps, identifying anti-bot defenses)
- Write and maintain python based data post-processors to transform raw HTML resources into standardised JSON models for downstream teams and systems to consume.
- Help monitor our distributed crawl jobs to ensure the team maintains its data coverage goals and reacts quickly to site changes or outages.
- Assist with testing and bug fixing new framework releases and related data qa tooling
- A BSc in Computer Science (or equivalent academic / self-taught achievement)
- Excellent communication skills: you are able to clearly articulate your actions and decisions. You must have a good command of the English language, both written and spoken.
- People are based in Portugal and who have the right to work in Portugal.
- CV and Covering Letter must be submitted in English language.
- Minimum 1 year professional software engineering experience (working within a team and releasing production code).
- Solid understanding of how websites are built and the core technologies involved (HTML, APIs, DNS, TCP/IP, CDNs, POST/GET, Proxies, Cookies, Headers).
- Basic experience with CI/CD tools, containers (Docker) and testing frameworks.
- Experience with basic production troubleshooting (code, deployment and execution)
- A curious mind and competitive spirit. You enjoy problem solving and finding clever ways to bypass obstacles put in your way.
- Calm under pressure and comfortable working in a fluid environment where priorities can shift and change regularly in response to external events outside of your control.
Bonus qualities that will really make you stand out from the crowd:
- Prior experience writing web scrapers (in any language) or defending against them.
- Familiarity with mobile app development (Android).
- Experience with AWS & its services.
- Experience running code in Kubernetes / EKS.
What you can expect from Daltix:
- Daltix believes it is powered by its people: You will be joining a team geared up for growth, giving you the unique opportunity to help shape our future as a company. Your day at work will matter. You will join a very special team; nimble, talented, international and highly motivated.
- We believe in flexibility at work. You will have a lot of autonomy in deciding what tasks are important and where you choose to work to keep you inspired. We trust you know your schedule.
- Transparency is our superpower. It means our organization is flat and our culture is open and inclusive.
- We inject fun into work. Virtual or in-person random catch-ups and spontaneous after-work fun is as common as informative monthly company-wide meetings and organized team events.
- Developing yourself is an important cornerstone of our company. Together, we will find relevant training to help you stay ahead of your, and our, game.
- A highly competitive wage with benefits such as health insurance, group insurance and meal allowance.