Context #
As CentOS 7 was approaching EOL, I converted our server into Alma Linux 8 around Easter. Everything seemed to be a great success and smooth, nothing changed, until we noticed this particular page on our WooCommerce store that producing Apache-native 404 error, whether accessing through the navigation menu, direct entry (via URL bar), or Google Search.
The error would present itself in two parts:
-
Create a redirect chain: instead of
example.com/category-1/product, the redirect chain turned intoexample.com/category-1/product/product/product/product/ -
Unable to reach the desired page and then return a 404 error with the redirect chain in the URL search bar.
The fact that the error immediately disappeared upon first page refresh, and only happen intermittenly made it much harder to troubleshoot.
Initial troubleshooting #
Identifying the issue with this page was complex. Since the redirect wasn’t specified by us, it had to be an automatic process somewhere in the tech stack. Specifically, possible causes from the lowest to the highest layer included:
- Database issues
- Apache misconfig in .htaccess rules
- Misconfigured manual redirects (unlikely, but possible given the number we set up during the site migration)
- Misconfigured permalink settings in WordPress
- Cloudflare causing issues (very unlikely but possible since it handles DNS lookups)
Troubleshooting these factors varied in difficulty. We started with the basics: checking manual redirects and permalink settings. We filtered over 1,300 manual redirects in our .htaccess file for the faulty URL, but found no conflicts. Refreshing permalinks didn’t help either.
Finally, requesting the HTTP headers from the browser pointed us toward the cause of the redirect.
This specifier indicates that WordPress (likely some core function) is causing the problem, so we started focusing efforts more on how WordPress handles rewrites and canonical URLs for pages.
Investigating WordPress’ rewrite and canonical functions #
WordPress has built-in mechanisms that automatically redirect to the ‘canonical’ version of a URL if there’s any ambiguity about the final URL. At first glance, the URL (example.com/product-category/product-type) appeared correct, but a backend function was causing a redirect to its canonical URL.
Debugging was enabled through the wp-config.php file, producing a debug log. However, WordPress doesn’t necessarily log canonical redirects, making this process challenging. Three WordPress templates could be responsible for causing intermittent 301 redirects:
- redirect_canonical
- template_redirect
- wp_redirect
redirect_canonical attempts to find the correct link when a user enters a non-existent URL based on an exact WordPress query. Even though the URL example.com/category-1/product exists, WordPress may interpret it differently. Debugging this in wp-includes/canonical.php didn’t yield helpful results since these hooks are site-wide, and issues likely stem from page configuration.
template_redirect was unlikely to cause random redirect problems, as it only specifies what template to load. Investigation here also didn’t yield results.
wp_redirect was a more likely source but still not a serious consideration, as most manual 301 redirects are specified through .htaccess. Since we created all 301 redirects during the migration to Wordpress last year, this was an unlikely cause but still worth checking.
In the end, these hooks had nothing to do with the intermittent 301/404 error.
Making progress #
Moving away from hooks and templates, it was crucial to understand how the page was being requested through WordPress.
HTTP headers didn’t tell the full story, especially with intermittent errors. Sometimes, requesting the page resulted in a 200 HTTP status, and everything loaded perfectly. Other times, the page either 404 with Apache’s default 404 page, or redirected so many times that the browser couldn’t even show the 404 page.
Query Monitor provided a detailed overview of how the page was requested and output to the browser, including the templates, hooks, scripts, styles, and database queries used. The goal was to compare this page to other cornerstone top-level pages in the navigation menu. Despite being created the same way, this page wouldn’t load properly under any circumstances.
By examining the queries that WordPress uses to fetch the page based on its slug and permalink structure, we aimed to find a fundamental difference in how the faulty page was requested compared to other top-level pages. These queries showed the requested slug, the matched regular expression rule, the matched query, and the query string.
Most of our websites usually have a complex permalink structure where each URL slug is prefixed by a “sub-folder” that is actually part of the URL slug. For example, top-level pages follow a structure like ‘/product-category/product-type/’. They should match the same type of query when requested.
The custom permalink structure seems logical:
/blueproduct-category/ = category
/product-type/ = postname.
So, why was it causing issues just for that particular page? At first glance, the structure didn’t seem problematic, but when the faulty page was requested, it appeared the permalink structure was causing issues.
When other top-level pages are requested, the queries show the matched query ignores the category element and only the pagename dictates how the page is requested.
This indicates that the permalink structure doesn’t dramatically affect other top-level pages — except for this particular product category page.
The matched rule and query for the faulty page were entirely different. WordPress was treating this page hierarchically, unlike any other page.
This discrepancy was the crux of the problem and finally provided a significant clue on how to resolve the error.
With the knowledge, the next step is to check if this issue was isolated. I pulled the request, matched rule, matched query, query string, and matching rewrite rules for each top-level navigation page.
All other pages were requested like normal, isolating the issue to the faulty page.
So my first thought was to adjust the permalink structure to omit the category, stopping the faulty page from being treated hierarchically. However, this create conflicts with the website’s knowledge center, complicating this approach.
The redirect chain showed WordPress was unhappy with the faulty page’s request, creating a redirect chain to reach the ‘final URL’. On the test site, the faulty page was requested as ‘product-type’, aligning with how the redirect chain operated. On the live site, however, the request was ‘product-category/product-type’, fundamentally different from the single-word requests of other pages.
Querying the database requests for loading the page showed that the faulty page was trying to load post meta from the wrong pages and didn’t include the correct WordPress post reference. This meant WordPress wasn’t querying the correct post ID, contributing to the redirect chain problem.
The solution #
Step 1: Delete and re-create the faulty page using the same permalink/URL structure:
Reasoning: There was a fundamental problem with how the page was requested. Updating post IDs through SQL queries wouldn’t fix this quickly. Deleting the page and its remnants from the database, then re-creating it, would provide a fresh start. A small error occurred during re-creation because a ‘/’ was missed in the custom permalink, but it was adjusted to avoid adverse effects.
Step 2: Ensure WordPress requests the page correctly:
Reasoning: Understanding how WordPress requested this page, the goal was to meet those requirements rather than force automatic processes to conform to our standards. The page was a product-category/product-type on the live site and product-type on the test site. The URL suffix was prioritized in the request, so the page needed to be pushed out as product-type.
WordPress uses two types of URL determiners: a custom permalink set in the post settings and the slug itself. The test site showed the slug for the faulty page as product-type, while the live site showed it as product-category. The test site worked, requesting the same post ID through the database. The live site slug was adjusted to product-type to match the regex request.
Step 3: Identify the database query requests and matched queries for the newly created and URL slug adjusted page
Reasoning: Using a query monitor, we discovered the types of database queries used when the page is requested. Re-creating the page with the same permalink structure created a refreshed connection between the post meta in the database and the page itself. The post ID matched the requested page correctly, and it was requested as product-type through the query.
Comparing this to another top-level page showed they were requested in an identical fashion.
Future consideration #
As of writing, the problem appears to be fixed. The error_log has been tailed to show all entries to this page and so far, there has not been a 301 or 404 request to this page since the suggested fixes were put into place.
It is very unlikely that this same error will be encountered in the future, but the permalink structure for this website is convoluted at best and will need to be closely monitored – especially if new pages are created.
If the error does arise, posing the same problem of a redirect chain, a new proposed fix of disabling URL guessing for redirects could be a decent solution.