Real browser requests
Pages are fetched from your browser, not through a proxy request from our server. You see 404 and 500 status codes, redirect chains and real response times exactly as your visitors do.
Site Health · Site spider (crawler)
Every page of your site, crawled from your own browser
The Türk SEM Website Crawler is a small Chrome and Edge extension that walks your pages one by one, exactly the way a search engine spider does. Up to 10,000 pages in a single run: real status codes, redirect chains, title and canonical signals; the results are stored, version by version, in your Türk SEM panel.
Can't use the store? Download the package and load it through developer mode instead.
- Version
- 0.1.1
- Package size
- 116 KB
- Browser
- Chrome / Edge 116+
- Technology
- Manifest V3
turksem.com/dash · Deep Crawl
Türk SEM SEO Crawler: Full Site Audit
v0.1.1 · Manifest V3 · 116 KB
Pages crawled
1.284 / 10.000
- 200/services/seo
- 301/blog/old-addressredirect
- 200/references2 issues
- 404/product/discontinuedbroken link
- 200/blog/technical-seo-guide
- 200/about-usmissing description
- 500/searchserver error
- 200/contact
Crawl in progress2 errors3 warnings
New
We made the crawl a few minutes longer
We were already walking your whole site. Since we open every page anyway, we decided to look at it with a security eye as well. Every page is now checked for content you never published, for hidden redirects and for queries that leaked into your search data. The crawl takes a few minutes longer. In return, you find out on the day the crawl finishes whether your site has been taken over, instead of months later.
How it crawls
It walks your site page by page, like a spider
The Website Crawler does exactly what a search engine spider does: it starts at the home page, follows every internal link it finds, and maps your whole site as a network. The map beside this text animates that walk: each node is a page, the lines are internal links, and the center is your home page.
- Starts at the home page and follows internal links recursively
- Visits each page only once (it never loops)
- Surfaces orphan pages that nothing links to
- Measures click depth page by page
Crawled pageBeing crawled nowIssue found
Why do we crawl through your browser?
Crawlers that run on a server see your site the way an outside client does. When the crawl runs in your own browser, pages are fetched under the same conditions a real visitor experiences.
No extra load on your server
The crawl runs on your machine and moves at your pace. There are none of the sudden request bursts, firewall blocks or rate-limit problems that cloud crawlers trigger.
With your own session
Because the crawl runs in your browser, areas behind a login, member-only pages and environments that aren't public yet can be crawled just as they are.
Results stay in the panel, versioned
Every crawl is stored as a version and compared automatically with the next one. No archiving files, no hunting for who ran which version.
How does it work?
- 1
Install the extension
Download and unzip the package, then load it in Chrome or Edge through developer mode. A one-off, two-minute job.
- 2
Pick your site in the panel
In your Türk SEM panel, open your site's Site Health section and select Deep Crawl. The extension is detected automatically.
- 3
Start the crawl
The extension starts at the home page and follows internal links to visit each page in turn. You watch progress live and can stop it whenever you like.
- 4
Read the report in the panel
Results flow into the panel: health score, errors, warnings, page-by-page findings and the difference from the previous version. It's also written into your monthly record.
What gets measured on each page?
The extension fetches the page, parses it in the browser and extracts the signals below. The parsing core is the same one our server-side analyzer uses, so both sides give the same result.
Response & redirects
- HTTP status code (200/301/404/500…)
- Real response time
- Redirect chain and chain length
- Page size (bytes)
On-page SEO
- Title and meta description
- H1 and the number of H1s
- Word count
- Content fingerprint (duplicate content)
Indexability
- Canonical tag (self-referencing / pointing elsewhere)
- noindex: meta robots and X-Robots-Tag
- Addresses blocked by robots.txt
- Click depth
Links & enrichment
- Internal link count (inlinks) and orphan pages
- External link count and target domains
- Images, and images missing alt text
- Open Graph and JSON-LD structured data
The report you see in the panel
As soon as the crawl finishes, the results turn into a report in your panel: health score, status code distribution, top issues, click depth and the difference from the previous version. No archiving files into folders; every crawl simply stays as a version.
Crawl Report
turksem.com · 25 Jul 2026 · Version 3
82
/100 health score
Status codes
20011733xx764xx235xx12
Top issues
1,284 pages · 12 errors · 47 warnings
- Missing meta description31
- Redirect chain23
- Click depth too great (>3)15
- Broken link (404)12
- Orphan page8
Click depth
10
421
6102
4183
2134+
A sample report; the numbers are illustrative. Your real report is generated from a crawl of your own site.
Where does it differ from desktop crawlers?
Desktop crawlers such as Screaming Frog are established tools in technical SEO and are strong at deep auditing. The Website Crawler focuses on a different need: repeating the same measurement every month, keeping the history in one panel, and combining findings with the site's other data.
| Feature | Türk SEM Website Crawler | Desktop crawlers |
|---|---|---|
| Setup | Chrome/Edge extension (2 min) | Desktop application install |
| Where the crawl runs | Your browser, with your own session | Desktop application, a separate client |
| Where results live | Permanently in the panel, stored version by version | A local project file; export by hand |
| Comparison with the last crawl | Automatic, the difference between versions | Usually a manual comparison |
| Sharing with your team | Through the panel account, by sending a link | File sharing or a shared license |
| Combining with other data | In the same panel as backlinks, Core Web Vitals and search data; the pattern engine connects them | Possible with connectors, but you build the setup |
| Where it's strongest | Continuity and context: the same measurement every month, the history in one panel | One-off, very deep technical audits and advanced filtering |
The comparison is based on general usage scenarios; the capabilities of desktop tools vary by version and license. Screaming Frog is a trademark of Screaming Frog Ltd and has no association with Türk SEM.
Installation
The extension is live on the Chrome Web Store, so it installs in one click. If you can't use the store, download the package below and load it through developer mode instead. Either way it's a one-off step, and you don't need to restart your browser.
Install from the Chrome Web Store (recommended)
One click is enough, and updates arrive on their own. The getting started guide opens as soon as the install finishes.
If you can't use the store: manual install
- 1Download the package (.zip)
- 2Right-click the ZIP and use “Extract All” to unpack it into a folder
- 3Open chrome://extensions
- 4Turn on “Developer mode” at the top right
- 5Choose “Load unpacked”
- 6Select the FOLDER you extracted
After installing: the step-by-step getting started guide
A common sticking point: Chrome's “Select the extension directory” window lists folders only; the .zip file you downloaded won't appear there. That isn't a fault; the folder created after you extract the ZIP will show up in the window.
Frequently asked questions
Does the Website Crawler cost anything?
No. Türk SEM membership and the Website Crawler extension are free; you can download and install the extension from this page.
What data does the extension access?
The extension only makes requests to your own site, and only for a crawl you started. It sends the technical signals of the pages it visits (status code, title, canonical, link counts and so on) to the panel. It collects no passwords, cookie contents or personal data.
Could it harm my server?
The crawl runs on your machine at a measured pace and obeys robots.txt rules. You can stop it at any moment.
How many pages can be crawled?
The Website Crawler walks up to 10,000 pages in a single run. The panel's server-side monthly crawl is limited to 150 pages; the extension isn't bound by that limit. How long it takes depends on the size of your site and the speed of your machine.
How does it differ from desktop crawlers like Screaming Frog?
The crawl runs in your browser, so no separate desktop application is needed and pages behind a login can be crawled with your own session. Results are stored and versioned in the panel rather than in a file, compared automatically with the previous crawl, and combined on the same screen with backlinks, Core Web Vitals and search data. Desktop crawlers, for their part, are strong at one-off deep audits and advanced filtering.
Is it on the Chrome Web Store?
Yes, it's live. Installing from the store takes one click and updates arrive on their own. If you can't reach the store, you can still download the package and load it through developer mode.
Are Safari and Firefox supported?
Chrome and Edge (Chromium-based browsers) are supported at present. Once a crawl has finished, you can view the report from any browser.
Crawl your entire site today
Free · Chrome & Edge · Up to 10,000 pages in a single run