Screaming Frog Review 2023

updated on January 11, 2023
Overall 5/5
Screaming Frog logo
Price
Accuracy
Features

Pros

  • Information Is Simple To Interpret
  • Accurate & Comprehensive Reports
  • Customization
  • Affordability
  • Efficiency
  • Cons

  • Limited to Windows & Mac OS X
  • Limited Free Version
  • Unattractive User Interface
  • Must Be Downloaded
  • Steep Learning Curve
  • Get a Free Trial Today

    Summary (5/5)

    Web crawlers, spiders, and bots, oh my!

    Though it may sound like the lyrics from a geeky horror musical gone terribly wrong, the world of search engine optimization (SEO) is actually seriously reliant on and invested in the reality of web crawlers.

    Web crawlers (also known as search engine bots or website spiders) are automated digital software applications that are programmed to find and index websites for search engines. They move from page to page, scrutinizing and reporting on on-page and technical aspects. Ultimately, their findings inform web page rankings.

    But wait. Wouldn't it be great if you could stay one step ahead of search engine bots by engaging a crawling tool of your own? A tool that could give you a snapshot of your site's strengths and weaknesses so that you could optimize it for future web crawls and boost your search engine rankings?

    Enter Screaming Frog.

    Screaming Frog is an SEO spider that crawls your site in search of on-page and technical SEO issues. It is incredibly thorough, straightforward, and built with digital marketing professionals in mind (consultants, agencies). I’m personally a huge fan of Screaming Frog. In fact, I'm not sure how I'd get through my day without it!

    Though no software is perfect, I stand solidly by my 5/5 star overall rating for the product.

    Features (5/5)

    Screaming Frog is fully loaded with features. From internal reviews to external, and from integrations to reporting, its intuitive interface and clearly organized tab and filtering systems keep everything in plain view. To my mind, Screaming Frog's features are refreshingly comprehensive and accessible. A winning combination.

    Accuracy (5/5)

    Across the board, Screaming Frog has a consistent reputation for reliability and accuracy. While no software can ever be 100% accurate all the time, Screaming Frog is about as thorough and trustworthy as it gets. Also, the software isn't glitchy, which is a huge plus in my books.

    Price (5/5)

    Screaming Frog offers a free version that may be adequate for the needs of small businesses. In my opinion, though, the paid version is well worth the approximately $200 annual fee (price is in British pounds, so fluctuates slightly with exchange rates). For the amount of benefit the program provides, I think its pricing is very reasonable.

    Introduction

    Screaming Frog is a spider tool that crawls websites to perform on-page and technical SEO audits. It identifies problems (missing meta descriptions, broken links, mismatched titles and tags, plus so much more) and shows you exactly what repairs are required and where. Using Screaming Frog to regularly crawl your website can keep it in tip-top technical condition and help it to attain and maintain higher rankings.

    Screaming Frog was founded in 2010 by a team of four experienced SEO professionals with proven personal track records. Why? Because they felt the digital marketing landscape lacked a trustworthy tool for collecting data, identifying technical issues, and providing a base for onsite SEO recommendations. Screaming Frog has certainly filled that void—and its sustained popularity is evidence of its success.

    Today, Screaming Frog has a more than 40-person team and is a staple tool for SEO professionals worldwide. Located in Henley-on-Thames, Oxfordshire, U.K., it's been showered with prestigious tech and marketing awards.

    As if all that weren't enough, I'll personally back Screaming Frog. As an experienced SEO professional for a leading agency, I can honestly say that I use Screaming Frog every single day. It’s one of my fundamental go-to tools and I’d be hard pressed to do my job without the efficient and relevant data it provides.

    That being said, Screaming Frog will not necessarily be the right tool for everyone. In this review, I'm going to walk you through the platform's pros, cons, and features. I'll finish off by looking at some case studies and answering a few commonly asked questions. If you stick with me, by the end you should have a good sense of what Screaming Frog has to offer and whether or not you want to invest in it.

    Let's get started!

    Screaming Frog Pros & Cons

    Pros

    • Information Is Simple To Interpret - It may not be beautiful, but Screaming Frog's interface is very straightforward and user-friendly. Data is displayed in four panes and you can navigate through a simple system of tabs and filters to find exactly what you're looking for.
    • Accurate & Comprehensive Reports - Screaming Frog generates incredibly detailed reports on a wealth of technical SEO factors. I’m continually impressed with the platform's accuracy and the depth of its data findings.
    • Customization - Screaming Frog allows users to customize crawl settings and create unique rules to suit their specific needs. This can help improve the accuracy and relevance of the data generated by the tool.
    • Affordability - Compared to many other on-page and technical SEO tools, Screaming Frog's annual subscription price is reasonable—not to mention, the product also offers a limited free version.
    • Efficiency - One of my favorite things about Screaming Frog is the fact that it doesn't waste my time. Average crawling speed is about 5 URLs per second, and you can minimize waiting time even further by limiting a crawl to specific pages or sections of your site.

    Cons

    • Limited to Windows & Mac OS X - Screaming Frog only works on Windows and Mac OS X, so you're out of luck if you use another operating system.
    • Limited Free Version - Screaming Frog's free version only supports 500 URLs, so it's not a realistic option for serious users. To be fair, though, the paid version is quite affordable for what it has to offer.
    • Unattractive User Interface - While Screaming Frog provides an organized and intuitive user experience, the aesthetics of its interface are not the best. Personally, I don't much care about the design as long as the tool is functional, but if a modern, elegant interface is important to you, you should probably look elsewhere.
    • Must Be Downloaded - Screaming Frog doesn't operate in the cloud. If you want to use it, you've got to download it. For some users, this may be a negative.
    • Steep Learning Curve - Although it's not difficult to use Screaming Frog to crawl your site, it can be challenging for beginners to know how to extract, interpret, and act on the platform's most salient data. This tool is best suited for those with some SEO experience already under their belts (which isn't necessarily a con).

    Screaming Frog Features

    Before I jump into my overview of Screaming Frog's features (of which there are many), let's just pause for a moment to talk about how they're laid out within the software. Essentially, Screaming Frog's main dashboard is divided into four panes.

    The top left pane shows crawl data organized in a format that resembles an Excel spreadsheet. At the top of the pane, all of the data is organized into separate tabs (Page Titles, Meta Description, Security, etc.) that can each be filtered and segmented to your heart's content. When you click on a tab, you'll then see its specific metrics reflected in the columns of the spreadsheet for each separate URL crawled.

    The top right overview pane updates a list of data in real time. You can scroll down through it to see which tabs are populated for any given crawl (rather than opening each tab individually). If you like, you can select the Issues Tab to see only a list of data that Screaming Frog believes to be problematic. (Be careful, though. Not everything in the Issues Tab is necessarily a problem you need to address. In some cases, intentional actions—such as no-index pages you've purposely created—may also get flagged.)

    The bottom left pane provides further information related to the specific URL and tab you've selected (URL details, Inlinks, Outlinks, Image Details, etc.). And, finally, the bottom right pane displays visual data graphs.

    At any point during a crawl, you can pause Screaming Frog, export information (narrowed down by tab and filter, if you wish), and then resume the crawl. You can also access historical crawls to note the progress being made as you make changes to a website (known as a comparison crawl).

    In case you require extra support, Screaming Frog offers a thorough selection of self-guided resources as well as technical support via email or a ticketing system.

    Now that I've provided a snapshot of Screaming Frog's architecture to put its features in context, let's get to them. Without further ado, here are some of Screaming Frog's core features (most of which are positioned as tabs):

    Internal Tab

    Screaming Frog's Internal Tab is the perfect starting point for analyzing the details of pages within your own website. Reported data includes Page Title, Meta Description, H1 and H2 Contents, Word Count, and much more. Available filters include HTML, JavaScript, CSS, Images, PDF Flash, Other, and Unknown. Worth noting is that, if you don't select a specific tab, the Internal Tab is Screaming Frog’s default tab.

    External Tab

    Not surprisingly, Screaming Frog's External Tab provides information about external URLs (URLs on a different subdomain than the page you started your crawl on). The External Tab provides less in-depth information than the Internal tab, but there's still plenty to work with. Reported data includes Address, Content, Status Code, Crawl Depth, and more. Filter options are the same as for the Internal Tab.

    Security Tab

    The Security Tab displays data related to security. It only displays security information for internal URLs. Reported data includes Address, Content, Status Code, Indexability, and more. Filters include HTTP URLs, HTTPS URLs, Mixed Content, Form URL Insecure, and more.

    Response Codes Tab

    The Screaming Frog Response Codes Tab will alert you in the event that any of your URLs are returning bad codes such as 404 (Not Found), 3XX (Redirect), or 5XX (Server Error). Reported data includes Address, Content, Status Code, Indexability, Response Time, Redirect URL, and more. Filters include Blocked by Robots.txt, Blocked Resource, No Response, and more.

    URL Tab

    This tab reveals common URL issues directly related to the specific URLs in your crawl. Reported data includes Address, Content, Status Code, Indexability, and more. Filters include Non ASCII Characters, Underscores, Uppercase, Multiple Slashes, and more.

    Page Titles Tab

    Page titles (also known as title tags or SEO titles) explain the purpose of a web page to users and search engines alike. They’re an important on-page SEO factor, and Screaming Frog helps to identify inconsistencies and other issues with them. Reported data includes Address, Occurrences, Title 1/2, Title 1/2 length, and more. Filters include Missing, Duplicate, Over 60 Characters, Below 30 Characters, Multiple, and more.

    Meta Description Tab

    Meta descriptions are those little write-ups you see below results on search engine results pages. Though they're not part of Google's ranking considerations, meta descriptions can still have a major impact on click through rates. Reported data for this tab includes Address, Occurrences, Meta Description 1/2, Meta Description 1/2 Length, and more. Filters include Missing, Duplicate, Over 155 Characters, Below 70 Characters, Multiple, and more.

    H1 Tab

    The H1 Tab takes your page's H1 tags under scrutiny. (There's also an H2 Tab that's very similar.) An H1 typically denotes a page's main heading, while H2-H6 represent smaller headings. Headings help human users as well as bots understand how page content is organized. They’re a strong on-page ranking signal. Screaming Frog's reported data for H1s includes Address, Occurrences, H1-1/2, Indexability, and more. Filters include Missing, Duplicate, Over 70 Characters, and Multiple.

    Content Tab

    The Content Tab is more or less Screaming Frog's version of a content editor. It reports on content elements that can impact your overall SEO. In addition to Word Count, Average Words Per Sentence, Duplicate Content, and Indexability, the Content Tab also includes data on readability, spelling and grammar errors, reading ease, and more. Filters include Exact Duplicates, Near Duplicates, Low Content Pages, Spelling Errors, Grammar Errors, Readability Difficult, and Readability Very Difficult.

    Images Tab

    In addition to crawling text content, Screaming Frog also crawls your image (and other media) links. The Images Tab reports on both internal and external images and reveals common issues related to images and their alt text. Reported data includes Address, Content, Size, and Indexability. Filters include Over 100kb, Missing Alt Text, Missing Alt Attribute, Alt Text Over 100 Characters, and Background Images.

    Canonicals Tab

    When multiple pages exist with the same URL (duplicate pages), you can identify one as the most representative page for ranking—the master copy of a page, if you will—by placing a canonical tag (rel="canonical") within the head of the document. Screaming Frog's Canonicals Tab reveals canonical link elements and HTTP canonicals that were turned up during a crawl. Reported data includes Address, Occurrences, Indexability, Canonical Link Element 1/2 etc., HTTP Canonical 1/2 etc., Meta Robots 1/2 etc., and more. Filters include Contains Canonical, Self Referencing, Canonicalised, Missing, Multiple, Multiple Conflicting, and more.

    Hreflang Tab

    Hreflang is, "an HTML attribute used to specify the language and geographical targeting of a webpage." It helps inform search engines about the relationship between business pages in different languages/regions and to ensure that the most appropriate page is displayed to the correct market segment. Screaming Frog's Hreflang Tab includes hreflang data such as Address, Title 1/2 etc., Occurrences, HTML hreflang 1/2 etc., Sitemap hreflang 1/2 etc., and more. Filters include Contains Hreflang, Non-200 Hreflang URLs, Unlinked Hreflang URLs, Missing Return Links, and more.

    Structured Data Tab

    In the world of SEO, structured data is the in-page markup that helps search engines understand page content. The Structured Data Tab in Screaming Frog includes structured data details and validation issues. Reported data includes Address, Errors, Warnings, Total Types, Unique Types, and more. Filters include Contains Structured Data, Missing Structured Data, Validation Errors, Validation Warnings, Parse Errors, and more.

    Sitemaps Tab

    Without an XML sitemap, a file that lists a website's most important pages, it can be difficult for search engines to find and understand its structure. Screaming Frog's Sitemaps Tab reveals sitemap information revealed during a crawl. Reported data includes Address, Content, Status Code, Indexability, and more. Filters include URLs In Sitemap, URLs Not In Sitemap, Orphan URLs, Non-Indexable URLs In Sitemap, URLs In Multiple Sitemaps, and more.

    PageSpeed

    To access the full range of data available within the Screaming Frog PageSpeed Tab, you'll have to configure the program to insert a free PageSpeed API key. Once that's done, Screaming Frog will pull data from Lighthouse. Reported data includes overview metrics such as Total Size Savings, Image Count, and Media Count, as well as insights based on both field data and lab data. You'll also have access to potential opportunities and diagnostics. Filters for this tab are plentiful and include Eliminate Render-Blocking Resources, Defer Offscreen Images, Efficiently Encode Images, Reduce JavaScript Execution Time, and many more.

    Connecting to Google Analytics & Google Search Console

    Google Analytics and Search Console integrations are only available with the paid version of Screaming Frog. If you opt to integrate the platform with GA and GSC (which I strongly recommend), you'll have to do the following:

    Connecting to Google Analytics

    To integrate Google Analytics with Screaming Frog, select it from the Configuration (Integration) menu and follow the steps provided.  

    Once integrated, Screaming Frog allows you to select up to 30 metrics at a time but automatically collects 10, which include Sessions, % New Sessions, New Users, Bounce Rate, Page Views Per Session, and more. Filters include Sessions Above 0, Bounce Rate Above 70%, No GA Data, Non-Indexable With GA Data, and Orphan URLs.

    Connecting to Google Search Console

    To integrate Google Search Console with Screaming Frog, select it from the Configuration (Integration) menu and follow the steps provided.  

    Once integrated, Screaming Frog collects data on clicks, impressions, CTR, and position. You can also opt to enable URL inspection alongside Search Analytics data. Filters include Clicks Above 0, No Search Analytics Data, Non-Indexable With Search Analytics Data, Orphan URLs, URL Is Not On Google, and more.

    Crawl Settings

    Different situations are best suited for crawls with different configurations. Fortunately, you can opt to adjust and customize Screaming Frog's crawl settings before unleashing the SEO spider tool on your website. I highly recommend experimenting with different settings and creating custom rules to find the configuration that best fits your specific goals and website structure as a whole.

    Screaming Frog provides a wide range of different crawl setting options. Here are a few of my favorites:

    • Maximum Crawl Depth - This setting determines how far-reaching a given crawl will be. You can decide the number of "clicks" away from your original URL that you want results included from (1 click, 2 clicks, etc.).
    • Maximum Crawl Duration - This setting allows you to put a time limit on a crawl. When the time limit is up, Screaming Frog will produce a report even if the SEO spider could have continued crawling.
    • Maximum Number of URLs - Similar to Maximum Crawl Duration, Maximum Number of URLs caps out a crawl when it reaches the number of URLs you've specified.
    • Include/Exclude Patterns - This setting gives you complete control over which URLs will and won't be included in any given crawl.
    • User-Agent - With this setting, you can program Screaming Frog to perform a crawl from the perspective of a specified user-agent (device type, browser, etc.). This can give you insight into issues and experiences that may vary based on how your site is accessed.

    Database Mode

    Screaming Frog's Database Mode allows big sites with more than 500,000 URLs to be crawled quickly and efficiently. How? By switching from memory storage to database storage. Memory storage is Screaming Frog's default setting and is fine for small and medium sites, but it basically uses your computer's RAM and can really slow down your whole system when overloaded. Using Database Mode speeds up crawls and can save you quite a bit of time and frustration.

    In Database Mode, you can automatically save crawls to your hard drive and pull them back up any time for reference and comparative analysis.

    Visualizations

    Screaming Frog offers access to two main types of interactive visualizations: Crawl visualizations (which provide a snapshot of the path the SEO spider uses to crawl a site); and directory tree visualizations (which reveal the URL architecture of a website). Both types of visualization can be accessed as either a force-directed diagram or a tree graph.

    Screaming Frog's visualizations don't actually include any additional data from what's available in the software's tabs, spreadsheets, and other reports. That being said, they provide a unique (and attractive) presentation of data that may allow you to notice new patterns and interpret information in fresh and unexpected ways. Personally, I find that visualizations can be particularly impactful when used in presentations or put forward for brainstorming and analysis sessions. Ask your team what jumps out at them and go from there!

    Reports

    Screaming Frog offers several types of data-driven reports based on crawls (accessible in the Reports menu). They cover everything from canonical errors to unlinked paginated pages. I've highlighted the three reports I use most often below:

    • Crawl Overview - As its name suggests, Screaming Frog's Crawl Overview Report basically summarizes a crawl in an easily digestible format. It includes information such as how many URLs the spider found and crawled, the types of content it encountered, and any response codes it ran into, plus much more.
    • Issues Overview - This report is focused on the problems and issues that Screaming Frog encountered during a crawl. As always, be sure to verify that what Screaming Frog views as an issue and what you view as an issue are in sync. The platform can do an incredible amount for you, but ultimately you've got to be the one calling the shots.
    • Orphan Pages - This report displays pages on your site that don't have any links to them, and thus weren't found during a crawl. (Screaming Frog knows they exist because they're visible via Google analytics.) Orphan pages can often occur after you've migrated your site, but are important to keep an eye out for at any point in time.

    Bulk Export

    Exporting crawl data from Screaming Frog is a great way to ensure that you can access it without having to open the tool itself. The process is straightforward and simple. You can export top window data (tabs and filters), lower window data (URL details, inlinks, outlinks, image details, and more), or you can bulk export data.

    The Bulk Export menu is located at the top of the Screaming Frog dashboard and includes many different categories and sub-categories. Screaming Frog will export all of the crawl data from whichever category (or categories) you select. Categories include Queued URLs, Link, Web, Path Type, Security, Response Codes, and many more. Sample sub-categories from Links include All Inlinks, All Outlinks, External Links, All Anchor Text, and more.

    Screaming Frog Accuracy

    Screaming Frog's SEO spider tool is an advanced and flexible SEO crawler that thoroughly audits the on-page elements of small and large websites alike. The software solution's algorithm is continually being honed and updated and the tool as a whole has an outstanding reputation for consistency and accuracy. Integrating Screaming Frog with Google Analytics and Google Search Console (paid version only) further boosts the accuracy of the data it is able to retrieve.

    When relying on a tool like Screaming Frog for the most accurate data possible, you want to be certain that it can handle anything you'll throw its way. And, overall, I think Screaming Frog delivers. It's nearly impossible to trip up the SEO spider.

    Customization

    As you've seen throughout this review, Screaming Frog is a highly customizable tool. Sometimes, data accuracy is dependent on knowing how to frame, extract, and interpret the exact information you're looking for, and Screaming Frog's customization options can assist you in doing exactly that. Using the program with finesse may take a little practice, but in the end, you'll find that you can customize just about everything to unearth accurate data with precision.

    Capable of Handling a Wide Range of Web Technologies

    Screaming Frog crawls websites with a huge range of coding languages and other technologies. From HTML to AJAX, JavaScript, and more, the SEO spider is prepared. Data accuracy remains a strength for Screaming Frog across the internet's many technologies.

    Capable of Handling a Wide Range of Content Types

    Text-based content, video content, audio content, image content, and more can all be handled by Screaming Frog's SEO spider. No matter how the content on your website is arranged or configured, the tool will be able to provide accurate data that can help you to improve its search engine optimization.

    Screaming Frog Pricing

    Screaming Frog has two prices: Free and paid. Though the free version is limited, it may be adequate for some users. At the very least, it can provide a free trial for you to familiarize yourself with the software before purchasing.

    Let's take a closer look at what the Screaming Frog free and paid versions have to offer:

    Free Version

    The free version of Screaming Frog has a crawl limit of 500 URLs, which is not much at all. (Depending on the website you're crawling, you could hit 500 URLs within one crawl.) That being said, it could be enough for users in certain situations to get some use out of it. Free users have access to 7 out of the 27 features Screaming Frog lists on its pricing page. One of the features not included is free technical support.

    I'd definitely recommend trying the free version before purchasing the paid version, but if you hope to use Screaming Frog in any serious way, I'd highly recommend investing in the paid version.

    The paid version of Screaming Frog costs approximately $209 per year for a single license, but the price per license decreases if you purchase multiple licenses annually. As I mentioned earlier, the cost of the software fluctuates based on the exchange rate of the British Pound.

    Paid Screaming Frog users have unlimited crawls (unless their own memory or storage is limited) and can access all features as well as free technical support.

    Case Studies

    Screaming Frog features 21 interesting case studies on its website, each one showcasing the impact that powerful SEO software combined with intelligent and bold digital marketing strategies can have on a business's success. I'd recommend taking a peek for inspiration if you have a few minutes to spare.

    As for me, I feel that I'm a bit of a case study myself. Because I actually use Screaming Frog each and every day for every client I work with, I feel that I'm uniquely positioned to talk about its impacts in my line of work (SEO and digital marketing). Using Screaming Frog to crawl the SerpFocus website itself, I was recently able to clean up some 4XX errors, redirects, and security issues. I would love the opportunity to show you what Screaming Frog could do for you in combination with my consulting services.

    Screaming Frog FAQ

    What Does Screaming Frog Do?

    Screaming Frog crawls websites to collect data on technical and on-page issues that need to be resolved to improve search engine optimization. Users can configure various tabs and filters to focus in on the most relevant data.

    Why Is Screaming Frog Called Screaming Frog?

    The name Screaming Frog was inspired by a frog that was trying to defend itself from two cats in the backyard of one of the platform's founders.

    How Do You Set Up a Crawl With Screaming Frog SEO, and How Do You Interpret the Results?

    For in-depth information on how to set up and use Screaming Frog, I'd recommend checking out the Screaming Frog User Guide. I'd also be happy to provide some pointers as part of my consulting services.

    Are There Any Limitations to Screaming Frog SEO, and How Does It Compare to Other SEO Tools?

    Overall, Screaming Frog is my top recommendation for site-crawling SEO tools. That being said, it’s limited to Windows and Mac OS X, must be downloaded, and has a pretty limited free version. Screaming Frog alternatives include Semrush , Xenu, Ahrefs, Sitebulb, DeepCrawl, and Wayback Machine (which is free).

    From The Author

    Terry Williams (fresh)

    Terry Williams

    For the past decade I've been building businesses and using SEO to rank them in top positions. For the last 5 years I've worked at a premier search engine marketing agency where it's my job to test and use SEO software, plugins, and strategies. Read my full bio here

    Subscribe
    Notify of
    guest
    0 Comments
    Inline Feedbacks
    View all comments
    Where you go for the 
    latest SEO Software Reviews
    © 2022 SERP Focus
    115 W 30th St #92802, New York, NY 10001, USA
    0
    Would love your thoughts, please comment.x
    ()
    x
    linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram