Our upgraded web crawler finds more pages and handles modern JavaScript websites. Train your agent on React, Vue, and Angular sites that simpler crawlers miss.
What's New
- JavaScript support - Renders SPAs and frameworks before extracting content
- Smarter discovery - Finds dynamically loaded links and navigation
- Cleaner extraction - Removes navigation and boilerplate, keeps main content
Get Started
- Go to Documents > Add Source > Website
- Enter your starting URL
- Set the maximum pages to crawl
- Click Start Crawling
Website Link Limits
| Plan | Links |
|---|---|
| Free | 10 |
| Hobby+ | Unlimited |