If you have ever browsed the internet, you have probably encountered a 404 error at some point.
A 404 error message can be helpful feedback for a web user, but using it in the wrong context can lead to a soft 404 error. A soft 404 error can negatively impact SEO performance, especially when a lot of them go undetected and they start to pile up.
To avoid this situation, developers should work closely with SEO experts to keep soft 404 error pages to a minimum.
Developers will want to work quickly. A soft 404 error can increasingly affect your website’s performance the longer it stays unresolved. In this guide, we will cover everything you need to know about soft 404 errors, from what the error is to how you can fix it, so that your site’s performance and rankings aren’t harmed.
What Is A Soft 404 Error?
A soft 404 error and a regular 404 error are often used interchangeably, but they mean different things.
The regular 404 error, sometimes just called a 404 error, occurs when a webpage is unavailable. In that case, a server sends the correct HTTP status code, which returns a message to display on the browser: “404 Not Found.” This code implies that the page cannot be found.
A soft 404 error, on the other hand, occurs when the server sends a “200 OK” status for the web page, but Google mistakenly interprets the message to read that a 404 error should be displayed. Although it is a mistake on the search engine’s part, it usually happens when the page looks like an error.
This is where the confusion arises. The search engine marks a page as a “404 Not Found” page, even though that’s not the webpage’s correct status. If you are experiencing soft 404 errors on your website and these errors are displayed in tools like Google Search Console, you should take an immediate step to fix them.
When a regular page gets a “200 OK” status, they are displayed in the search engine results. They can be crawled and indexed. However, pages with soft 404 errors will also appear in the search engine result and can even be indexed or crawled, unlike a true 404 error, which are neither indexed nor displayed in results.
What Is An Example Of A Soft 404 Error?
When a user searches for something on your website that does not exist, they will get a “Not Found” message, which a search engine may misinterpret. This is what happens in a soft 404 error.
With a soft 404, the server response code is “200 OK,” but the search engine still thinks that there is a query on your page. Therefore it erroneously interprets the page as a 404 error.
If you are trying to access a page that exists in the CMS, but is still not being displayed by Google, chances are it is due to a soft 404 error.
If the category page does not have any content under the category, it will likely lead to soft 404 errors. Even when the webpage exists on the website, if there is no content, Google will interpret the page as a blank and displays a 404 error.
Causes Of Soft 404s
Several situations can create soft 404s for Google. Some of the most notable causes of this error include:
The Page Has Little To No Content
This is one of the primary causes of a soft 404 error. If the page has thin content or no content at all, Google expects results pointing to a status code of “404 Not Found.”
Even when the status response is okay, Google may still display the 404 error. Some examples of a soft 404 error due to lack of content are empty product category pages, empty blog category pages or empty search result pages.
Fixing these pages or adding content to them may help you get rid of the 404 error. However, if that hasn’t solved the problem, applying the “no index directive” through the meta robots tag can be a solution.
Redirect Target Is Not Relevant Enough
If you are redirecting your URL to another URL, it must be relevant, or Google will display a “404 Not Found” message. This issue primarily occurs when developers try to redirect users from one webpage to another that is not relevant. This issue is prominent on eCommerce websites, which redirect a user to another product or category not relevant to Google’s algorithms.
You Have Accidentally Blocked Google From Rendering
This issue can be resolved by debugging and verifying your webpage via Google’s inspection tool. After fixing this issue, Google will be able to render your files and the 404 should be resolved.
Page Content Has 404-like Phrases
If your page has phrases that are primarily found on a 404 page, it can cause a soft 404 error.
Avoid phrases you would find on a 404 error page, such as “not in stock,” “does not exist” and “no longer available.” These phrases can be mistaken for a 404 page by Google. Removing them from your site can fix the error.
The Difference Between A 404 Not Found And A Soft 404
The primary difference between a “404 Not Found” error and a soft 404 error is the status code. The former has a status code of 404 or 410, which implies that the page cannot be found.
With a soft 404 error, the page is still not found. But instead of returning the 404 or 410 code, a “200 OK” code is returned.
404 pages are not indexed by search engines and do not appear in the search engine results. However, soft 404 pages are indexed by search engines and can appear in the search results.
How Google Sees Soft 404s
Google has changed the way it handles soft 404 detection and classifications.
Google now looks at each page by device type and assigns a soft 404 classification differently to the same URL on desktop and mobile. If Google sees a URL and accesses the same URL on desktop and mobile, it may return a soft 404 error on mobile and desktop, or vice versa.
This means Google now detects a soft 404 status on a URL, as it goes through a URL. It also does so by device type.
This can be a problem when the page works correctly on a mobile Search Console, and does not throw any alerts. Meanwhile, some of your pages may be experiencing soft 404 errors outside of the console, which can affect your website’s performance and rankings.
Why Soft 404s Are Bad For SEO
If a website returns a status OK code rather than a 404 for a page that does not exist, it can affect the website’s performance negatively in an organic search.
Therefore, if your website has high soft 404 errors, you will be negatively impacted in search results. With a soft 404 error, the website will indicate that there is a real page at the URL and that visitors can access it. Search engines will index and crawl a page that does not actually exist, draining your valuable crawl budget on non-existent pages.
Googlebot only wants to crawl a limited number of websites and it divides its efforts by crawl rate and crawl demand.
- Crawl Rate – This allows Google to not crawl pages that are too fast and can hurt the server.
- Crawl Demand – This implies the pages Google wants to crawl. This budget is based on the popularity of your pages and how old the content is. The search engine does not want to waste its time crawling content on the same website. Google assigns a budget to its web crawls before it moves on to the next website.
If your website has high soft 404 errors, the crawl budget may be spent on pages that reduce your visibility. Visitors looking at search results will not be able to access pages with actual content. A high crawl percentage also affects the performance of your website in organic search.
By reducing your soft 404 errors, you increase your site’s visibility and search performance.
How To Find Soft 404 Errors
Finding a soft 404 error is not particularly challenging. An easy way to discover the errors is to log into Google’s Search Console and check for 404 errors. Once you have identified a 404, you want to make sure that it is returning a 404 error code.
But if they are not, you should fix them immediately. These are your soft 404 errors.
Screaming Frog is another tool to find broken links within your website. It also identifies websites that have linked to pages that no longer exist. Another alternative to Screaming Frog is Xenu Link Sleuth.
How To Fix And Resolve Soft 404s
Not all 404s are bad, as there are cases when they are displayed in the correct context. When a product is not available, for example, displaying a 404 error page lets users know that the particular product is permanently removed from the website.
Soft 404 errors can be tricky, however. There can be cases when a webpage is not valid and a server still returns a “200 OK” status. In these cases, you should look at the errors and try to fix them.
Here are some solutions for addressing soft 404 errors.
Check If The Page Is Indeed A Soft 404 And Not A False Alarm
Check for false alarms. In most cases, a page incorrectly marked as a soft 404 error by Google Search Console can be audited and verified.
- Start by clicking on the “Submitted URL seems to be a soft 404” button from the Coverage Report. This returns a complete list of soft 404 pages.
- Open the URLs in new tabs.
- If the page is a valid part of your website and you want it to appear in the search results, choose the “Validate Fix” option. This will make Google crawl the page and update the status code.
- Once complete, you should inspect the page and test the live URL.
An alternative method is to select the URL and click on the “Inspect URL” option. This will give you more information about the page of interest and give you an option to “Request Indexing.”
Before doing so, test the live URL to allow Google to refresh its report. Doing this will give you the correct status of the page. In most cases, the page will function correctly and require no changes.
Improve The Affected Page And Request Indexing
If your page exists, but Google has been reporting it as a soft 404, it may be due to the page content. In this case, improve the page content and resubmit it to Google.
This is a common problem when there’s thin content. Adding more content to the page is an easy fix. Once updated, the page will become more crawable to Google, helping to eliminate the soft 404 error.
Keep The Page On Your Site But Try To De-index It From Search Engines
Another alternative to resolving a soft 404 error is to keep the page on your site, but deindex it from the search engine.
Adding a no-index directive in the header will instruct the search engine not to index that particular webpage on the site. Doing this will allow you to fix the soft 404 error, since Google will not display the page under the Error report.
Note, you will still see this page listed in the excluded report under the Soft 404 section.
Configure Your Server To Return The Proper Not Found Error Code (404/410)
If the page is not available or invalid, you should try to configure the website to return the correct status code. Once you have configured your website for the correct code, resubmit the page to Google for indexing. It should help you get rid of the soft 404 error.
Configuring your site to return a 404 code for invalid pages includes deleting the pages. Once you delete a page, the HTTP server will show a 404 code when the page is requested, allowing you to reduce your soft 404 percentage.
Redirect The Page Using A 301 Redirect
The last method to resolve the soft 404 error is by redirecting the page of interest to a valid page. You can do this by adding a 301 redirect into your .htaccess file.
This tells the search engine that the page is moved to a new location and ensures it does not display the 404 error for an existing page. Be sure to check that both of the pages have similar content before you redirect one to another.
A Word About Website Maintenance And SEO
A correct 404 error is valid and lets visitors know about pages that have been permanently deleted. However, soft 404 errors can affect the performance of your website because they’re frustrating for users and confusing for search engines.
You should keep your soft 404 percentage to a minimum to make sure that errors do not affect your search visibility and rankings. Regular site maintenance is the easiest way to ensure that soft 404 errors don’t negatively affect your site.
Frequent site check-ups will allow you to keep errors at bay and make your website accessible for visitors. Consider enlisting the help of a company trusted by Google itself to maintain your site.
Prerender can help you unlock your website’s true potential. Follow us on Twitter for more technical SEO information to keep your website running at peak performance!