404-error, it is Not Found (from the English. "not found"), is the server response code that indicates that it is available for the client to access, but the requested data cannot be found.
The 404 error is one of the most common. Users see it by clicking on links that lead to documents that don't exist at the moment. As a result, the user does not see the expected information and, as a rule, immediately leaves the site, which adversely affects the behavioral factors of ranking and loyalty to the resource.
Mechanism and causes
Error 404 can occur when:
- the user entered the page address incorrectly;
- The page has been removed from the site.
- the document was moved to a new address, but did not make a redirect (redirect).
Before opening the contents of any page, the user's device sends a health request to the web service. In response, the server sends the browser an HTTP header with the page status. Normally, this is "200 OK". If the desired material is not found, the server returns an error message "404 Not Found", and the user sees it instead of the desired page.
Each digit of the code contains information: the first four indicates that a client error occurred, zero indicates a syntax error in the URL, and the final digit four means that the requested page is not on the server.
How to get rid of it
First, you need to find all documents with this server response code. To do this, you can:
- view the upload of indexed pages in Webmasters Tool and upload a list of URLs with the detected error 404;
- use any program that allows you to scan the site for errors (Xenu, Screaming Frog SEO Spider, etc.).
If the pages have been completely removed from the site, but users from search engines come to them, you can manually send a request to exclude them from the index. Usually this is not required, non-existent documents themselves fall out of issuance, but there are exceptions when intervention is needed.
In Yandex.Webmaster you need to add the list of pages with 404 error in the special field “Delete URL” in the “Tools” section.
In the Google Search Console, you need to specify documents one by one in the "Delete URLs" section in the "Google Index" section.
It is important to make sure that there are no deleted files in the XML sitemap.
If the content was not deleted, but changed addresses, you need to configure redirection from the old to the new ones. The 301 redirect is used for this purpose. Documents will be automatically replaced in the search results after reindexing and updating the search database.
To speed up the process, with a small number of addresses, you can take the opportunity to send the search bot for a crawl. In Yandex.Webmaster, this is the “Indexing” - “Page Crawl” section. The limit is determined individually for each site. For some, it can be 30, 60, 80 URLs.
In the Google Search Console, this is the “Crawl” - “View as Googlebot” section, and then click on “Request Indexing”.
If there are a lot of URLs, you need to send a rebuilt site map to the crawl.
What to do with internal links? It’s better to fix them anyway, because even if you configure redirects, it’s not very good when there are a lot of them.
If the content was deleted, broken links appear. You need to find them using scanning programs and update, delete, or replace them with suitable ones.
If there are external links to pages with the 404 error, you need to change them if possible, and if they are purchased, you need to remove them.
To minimize the possible harm caused by a user being redirected to a non-existent page, you need to configure it correctly:
- be sure to make the original design for the 404 error in the style of the site so that there is no standard "stub";
- explain that an error occurred, specify a link to the main page, search, popular categories, or other useful materials;
- you can write contacts to contact the webmaster.
Sometimes site owners turn the 404 page into a marketing tool or a viral feature.
For example, they offer a discount on a promo code to those who saw the 404 error, or post a funny image, animation, and so on.