Member of Technical Staff (Web Access Engineer, Search Crawler)
💰 $80,000 – $130,000/yrMarket estimate · not provided by the employer
At a glance
- Who can apply
- United Kingdom, Serbia, Ireland
- Work arrangement
- Hybrid · Remote
- Employment type
- Full-time
- Experience
- 5+ years (as stated by the employer)
- Salary
- $80,000 – $130,000/yr · market estimate
Checked against the employer's source on November 21, 2026. Details come from the employer's posting; confirm them on the application page.
Advertisement
Put your brand in front of 300,000+ monthly visitors worldwide
Job Description
Perplexity is hiring a Web Access Engineer for the Crawler team to build reliable fetching and rendering systems for AI-powered search. You'll work on freely available public web content, within site permissions and approved access policies, and develop the infrastructure that supplies content to downstream indexing and retrieval systems.
Your focus will be diagnosing network errors, page-loading issues, and incomplete rendering. You'll trace problems from network protocols through client-side execution, distinguish technical failures from intentional access restrictions, and turn findings into production fixes that improve content quality and efficiency at scale.
Responsibilities
Own fetching and rendering improvements end to end, from investigation through implementation and production rollout, partnering with crawler engineers and SREs.
Investigate fetch and rendering failures across DNS, TLS, HTTP, caching, redirects, and client-side rendering, and classify root causes.
Improve page loading, completion detection, and content capture to preserve relevant content and handle dynamic pages reliably.
Ensure the crawler respects site-imposed access restrictions and rate limits, identifies itself clearly, and retries only transient technical errors.
Build tools to reproduce failures, compare crawler output against expected page content, and turn findings into regression tests.
Define metrics for fetching reliability, content completeness, and error classification, and use them to prioritize improvements.
Qualifications
5+ years of software engineering experience in web infrastructure, browsers, CDNs, or crawling, with strong programming skills in at least one of Go, Python, Rust, or C++.
Deep understanding of HTTP/1.1, HTTP/2, and HTTP/3, including status semantics, caching, conditional requests, content encoding, and redirects; strong knowledge of TLS, DNS, cookies, and sessions.
Understanding of CDN caching and content delivery, HTTP error handling, retry backoff, and server-defined rate limits.
Knowledge of page lifecycle, network idle, service workers, client-side rendering, and shadow DOM; experience debugging JavaScript and fluency with Chrome DevTools, CDP, HAR files, curl, and packet captures.
Comfort working independently when the path forward is not fully defined.
Nice to have
Experience developing CDN, caching, or web performance systems.
Experience operating crawlers, headless browsers, or distributed content-acquisition systems at scale.
Contributions to a browser engine or JavaScript runtime.
Advertisement
Put your brand in front of 300,000+ monthly visitors worldwide
Related Jobs
Find more than a single listing
Discover more jobs
Join now and find your dream job or a higher-paying opportunity with better employer visibility, stronger internal discovery and easier access to worldwide hiring signals.
Discover hidden jobs
Surface roles you may miss on noisier boards and move through cleaner employer links and better intent pages.
Head start against the competition
See fresh roles, related pages, salary context and company signals faster so you can apply earlier and better.
10,000+ more jobs than narrower job boards
Explore remote, hybrid and location-based opportunities across worldwide markets instead of relying on one narrow inventory.
Worldwide opportunity coverage
Use region pages, country-eligibility pages and skill pages built for global job seekers, not just one market.
Social proof
Trusted by professionals
“The search experience was straightforward, although more filters would be helpful.”
Ernesto Machava
Data Engineer · 🇲🇿 Mozambique
“I found an international opportunity that closely matched my experience.”
Amélia Mucavele
Business Analyst · 🇲🇿 Mozambique
“There are good vacancies, but availability varies depending on the region.”
Cheryl Tan
Operations Analyst · 🇸🇬 Singapore