Erlang Developer

Scrapinghub
๐Ÿ™ 87 locations
๐ŸŒŽ 27 countries
๐Ÿค“ 51-200 members

Crawlera is a smart downloader designed specifically for web crawling and scraping, removing the headaches of proxy management. It is part of the Scrapinghub platform, the worldโ€™s most comprehensive web crawling stack which powers crawls of over 8 billion pages per month.


As an Erlang developer you will help to ensure the robustness of our services. You will learn to investigate production issues on a server executing customer requests. You will be able to navigate a large code-base and find the least obstructive place for extensions. Beside the technicalities you will gain a holistic view of the product and ensure a greater usability of the system with every single task you complete. In this role, you will partake in brainstorming and delivering improvements to the core of Crawlera.


Job Responsibilities:



  • Develop, maintain and support a high load distributed system.

  • Analyze our current and historical Crawlera usage to augment and enhance its routing and rotation logic.

  • Leverage the Scrapinghub platform to provide extended functionality, both to end users and for internal purposes.

  • Identify and resolve performance and scalability issues with distributed crawling at scale.

  • Liaison with other platform teams to provide Crawlera with the best possible integration to the growing Scrapinghub platform.


Job Requirements:



  • 2+ years of production experience with Erlang.

  • Strong communication in written and spoken English.

  • Strong knowledge of Linux/UNIX, HTTP and Networking.


Desired Skills:



  • Python or Golang experience.

  • Familiarity with techniques and tools for crawling, extracting, and processing data.

  • Knowledge of ELK, Graylog, Docker and Mesos.

  • Strong record of open source activity

  • Experience working with Lean principles and a Scrum SDLC

Benefits

Workplace

Tools used for remote work

Remote team at Scrapinghub is using 15 tools like World Time Buddy, OfficeVibe and Hangouts when working remotely.

Engineering

๐Ÿ”“ Open source
Paid open source work

About

Headquarters

Scrapinghub's remote company is headquartered in County Cork, Ireland

Remote team in 87 cities and 27 countries

Scrapinghub is working remotely from 87 cities like Cork, Dublin and Tokyo across 27 countries like Brazil, India and Russia with the average temperature of 20ยฐC.

๐Ÿ‡ง๐Ÿ‡ท ๐Ÿ‡ฎ๐Ÿ‡ณ ๐Ÿ‡ท๐Ÿ‡บ ๐Ÿ‡ช๐Ÿ‡ธ ๐Ÿ‡ฎ๐Ÿ‡ช ๐Ÿ‡บ๐Ÿ‡พ ๐Ÿ‡บ๐Ÿ‡ฆ ๐Ÿ‡ต๐Ÿ‡ฐ ๐Ÿ‡ต๐Ÿ‡ฑ ๐Ÿ‡ญ๐Ÿ‡บ ๐Ÿ‡ท๐Ÿ‡ด ๐Ÿ‡จ๐Ÿ‡ณ ๐Ÿ‡ฑ๐Ÿ‡ฐ ๐Ÿ‡ฏ๐Ÿ‡ต ๐Ÿ‡ฉ๐Ÿ‡ช ๐Ÿ‡ฎ๐Ÿ‡ฉ ๐Ÿ‡บ๐Ÿ‡ธ ๐Ÿ‡ฆ๐Ÿ‡ท ๐Ÿ‡ฌ๐Ÿ‡ง ๐Ÿ‡ต๐Ÿ‡น ๐Ÿ‡จ๐Ÿ‡ฟ ๐Ÿ‡ฒ๐Ÿ‡ฆ ๐Ÿ‡ง๐Ÿ‡ฉ ๐Ÿ‡ต๐Ÿ‡ญ ๐Ÿ‡ต๐Ÿ‡ช ๐Ÿ‡ง๐Ÿ‡ฌ ๐Ÿ‡ป๐Ÿ‡ช