On-site - CRAWLABLE Factor
A JSON containing data about all the robots technology found on the given website page. Please note that each robot can be configured differently, like on the example below which is configured to crawl all the page. If any robots crawler was found or the website content could not be load, then returns null.
{
"robots":[
"all"
],
"googlebot":[
"all"
]
}
| Key | Type | Description |
|---|---|---|
| robots | Object | Googlebot crawler found on the website page URL and how it is configured. |
| googlebot | Object | Googlebot crawler found on the website page URL and how it is configured. |
| googlebot_news | Object | Googlebot News crawler found on the website page URL and how it is configured. |
| googlebot_images | Object | Googlebot Image crawler found on the website page URL and how it is configured. |
| slurp | Object | Slurp crawler found on the website page URL and how it is configured. |
| msn_bot | Object | MSN bot crawler found on the website page URL and how it is configured. |
| teoma | Object | Teoma crawler found on the website page URL and how it is configured. |
| robots_header | Object | Robots Header crawler found on the website page URL and how it is configured. |
More SEO Factors
- Google Adsense
- Alexa Global Rank
- amp
- Google Analytics
- Backlinks Anchors
- Backlinks
- Blog Page
- Citations Number
- Conversion Forms
- crawlable
- Doctype
- Duplicate Content
- Emails
- Custom 404 Page
- Facebook Shares
- Facebook Page
- Favicon Image
- Flash
- Frames
- Google Preview
- Google Page
- Google My Business Rating Score
- Google My Business Reviews Number
- Google Plus shares
- Headings
- HTML validity
- Alternative Text for Images
- Language
- Linkedin shares
- load-time
- Logo
- Meta description
- Meta Keywords
- mobile-check
- NAP as TEXT
- on-bing-maps
- Registered to Google Maps
- In-Page Links
- Phones
- Pinterest images
- Google My Business +1es Number
- Printer Friendly CSS
- rel-author
- Google Canonical URL
- Google Publishership
- Responsiveness
- robots.txt
- security-observatory
- IP Address
- Location
- XML SiteMap
- Speed Score
- SSL Http Check
- Structured Data
- Technologies
- Encoding
- Text/HTML ratio
- thumbnail
- Title
- Twitter recent backlinks
- Twitter Page
- URL
- url-protocol
- URL Underscore
- WWW resolve