Diagnose the response and content

Inspect the URL with a direct HTTP request and Search Console URL Inspection. Compare the status code, rendered body, canonical tag, robots directives and internal links. A page can return 200 while displaying a generic not-found message, an empty product shell or a JavaScript error.

Segment affected URLs into never-valid, permanently removed, temporarily unavailable and still-useful. Do not apply one redirect rule to all four groups; the user intent and replacement options differ.

Choose the correct treatment

For a permanently removed page without a close replacement, return a real 404 or 410 and remove internal links and sitemap entries. If a successor genuinely serves the same need, use a relevant permanent redirect rather than sending every obsolete URL to the homepage.

For temporarily unavailable products, keep a useful 200 page with clear availability messaging, alternatives and accurate structured data where applicable. A service page that remains useful should retain its substantive description even when bookings are temporarily paused.

Audit templates and edge cases

Check category filters, pagination, expired campaigns, product variants and client-side routing. Many soft 404 patterns come from shared templates that render the same thin placeholder across thousands of URLs. Verify both raw and rendered output on representative samples.

Fix the source template before requesting indexing again. Keep a list of affected patterns, representative URLs, expected statuses and the date of deployment so follow-up checks measure actual resolution.

Verify recovery

After deployment, crawl the affected URLs and confirm expected status codes, canonical destinations, sitemap membership and navigation links. Search Console reporting may lag, so combine its trends with server logs and targeted URL inspections.

Measure restored discovery and qualified organic landing-page sessions rather than treating a declining soft-404 count as the only outcome. A truthful removal can be the correct result even if indexed URL totals fall.

Practical example and acceptance criteria

Consider a discontinued product with a popular informational page. If the page still answers maintenance questions and offers relevant alternatives, retaining a useful informational URL may serve visitors better than deleting it. If the content only says unavailable, an accurate removal or replacement may be appropriate.

A common failure occurs when an application returns 200 for every unknown route and renders a branded not-found screen. Test random nonexistent URLs, deleted IDs and expired promotional links to find the template-level issue.

Do not block a URL in robots.txt before search engines can see a 404, 410 or noindex directive. Verify the intended crawl and index path rather than stacking conflicting signals.