Staff Data Engineer
Slack is looking for an Staff level Data Engineer to help design and scale our batch and streaming data pipelines to help our engineers, operations team, product managers, and analysts make better decisions with data. You will build scalable backend services and tools to help partners implement, deploy and analyze data assets with a high level of autonomy and limited friction. You will play a meaningful role in making partner interactions with the Data Warehouse pleasant and productive (Analytics, Business Intelligence, Application Engineering, Machine Learning and IT). As Slack’s data grows (along with the number of customers, features and employees), the goal of the Data Platform team is to strengthen the efficiency and dependability of the way we make decisions. You will design and build abstractions that hide the complexity of the underlying Big Data stack (Hadoop, Hive, Spark, Presto, Kafka, Parquet, Airflow, EMR, S3, etc) and allow partners to focus on their strengths: data modeling, data analysis, search or machine learning.You will have deep technical skills, be comfortable contributing to a nascent data ecosystem and building a strong data foundation for the company. You will be a self-starter, detail and quality oriented, and passionate about having a huge impact at Slack.
Slack has a positive, diverse, and supportive culture—we look for people who are curious, inventive, and work to be a little better every single day. In our work together we aim to be smart, humble, hardworking and, above all, collaborative. If this sounds like a good fit for you, why not say hello?
What you will be doing
- Optimize the end-to-end workflow of data users at Slack (from crafting libraries to providing abstractions used to define jobs, schedule data pipelines or access data assets).
- Provide visibility into our data flows (comprehensive view of sources, transformations, sinks, data lineage).
- Automate and handle the lifecycle of data sets (schema evolution, metadata store, change and backfill management, deprecation and migration).
- Streamline the creation of new data sets with accessible frameworks and Domain Specific Languages (DSL).
- Improve the data quality and reliability of the pipelines (monitoring and failure detection).
- Supply reusable backend abstractions to ingest or access data sets (batch or low latency APIs).
What you should have
- Bachelor's degree in Computer Science, Engineering or related field, or equivalent training, fellowship, or work experience.
- 5+ years of experience working with Big Data technologies (e.g. Hadoop, Hive, Spark, Presto, Kafka, etc).
- Deep understanding of polyglot data persistence (relational, key/value, document, column, graph).
- Skilled at crafting and building robust backend data services (distributed systems, concurrency models, microservices).
- Strong dedication to code quality, automation and operational excellence: unit/integration tests, scripts, workflows.
- Expertise in object-oriented and/or functional programming languages (e.g. Java/Scala, Python).
- Excellent written and verbal communication and social skills; able to effectively collaborate with partners.
Here are a few extra things that would lift you up a couple of notches in our eyes:
Academic background in computer science or mathematics (BSc or MSc)
Experience with SOLR and Redis
Experience building simple scripts and web applications using Python, Ruby, or PHP.
A solid grasp of basic statistics (regression, hypothesis testing)
Experience in small start-up environments
Slack is a layer of the business technology stack that brings together people, data, and applications – a single place where people can effectively work together, find important information, and access hundreds of thousands of critical applications and services to do their best work. From global Fortune 100 companies to corner markets, businesses and teams of all kinds use Slack to bring the right people together with all the right information. Slack is headquartered in San Francisco, CA and has ten offices around the world. For more information on how Slack makes teams better connected, visit slack.com.
Ensuring a diverse and inclusive workplace where we learn from each other is core to Slack’s values. We welcome people of different backgrounds, experiences, abilities and perspectives. We are an equal opportunity employer and a pleasant and supportive place to work.
Come do the best work of your life here at Slack.