- The core areas of technical SEO and how they affect indexability and rankings
- How to diagnose and fix the most common technical issues on Indian fintech sites
- Best practices for robots.txt, XML sitemaps, HTTPS, and mobile-first indexing
- A step-by-step technical audit process you can run on any website
What Technical SEO Covers
Technical SEO is the discipline of ensuring search engine bots can efficiently crawl, render, and index your website — and that the resulting experience for both bots and users meets the quality bar Google expects. It's the invisible infrastructure that either amplifies or undermines everything else you do in SEO. The best content in the world cannot rank if Google can't access and understand it.
Unlike on-page SEO (which you can fix page by page) or link building (which happens gradually), technical SEO issues can affect your entire site at once. A single misconfigured line in robots.txt can de-index thousands of pages overnight. This makes technical SEO both the highest-risk and the highest-reward area of the discipline — fix one critical issue and unlock rankings across hundreds of pages simultaneously.
Crawlability and Robots.txt
The robots.txt file is a plain text file at the root of your domain (e.g., bankbazaar.com/robots.txt) that tells crawlers which parts of your site they can and cannot access. A correctly configured robots.txt protects admin pages, duplicate content, and staging environments from being crawled — while ensuring all your important content pages are fully accessible.
The danger zone: many technical SEO disasters come from robots.txt misconfigurations. The classic example is a developer accidentally blocking all crawlers (Disallow: /) during site migration — and forgetting to revert it on launch. This happened to a major Indian comparison site (not BankBazaar, but a competitor) and cost them 3 weeks of ranking recovery. Always test your robots.txt with the URL Inspection tool in GSC before making changes.
During a platform migration, a financial comparison site's new robots.txt blocked the entire /products/ directory — their highest-traffic section. Google respected the block within 48 hours, de-indexing 2,000+ pages. The team discovered the issue via a sudden 60% traffic drop in GA4 and a flood of "Blocked by robots.txt" errors in GSC. Fix: corrected robots.txt, submitted emergency re-crawl request via GSC. Full recovery took 18 days. Lesson: Always run a robots.txt check as part of any migration QA checklist.
XML Sitemaps
An XML sitemap is a file that lists all the pages on your site you want Google to crawl and index. It's particularly valuable for large sites (500+ pages), sites with complex navigation, or newly published content you want indexed quickly. Submit your sitemap via Google Search Console at GSC › Sitemaps — Google will then use it as a crawl guide, though it's a suggestion, not a command.
For a fintech site like PolicyBazaar, multiple sitemaps are best practice: one for insurance product pages, one for blog articles, one for city/branch landing pages, and one for calculators/tools. Group pages logically and exclude pages you don't want indexed (thin pages, duplicate content, paginated pages beyond page 2). Update your sitemap dynamically whenever you publish new content — use your CMS's auto-sitemap feature or a plugin like Yoast for WordPress sites.
Site Speed and Performance
Page speed is both a direct ranking factor (especially for mobile) and the primary driver of Core Web Vitals — Google's user experience metrics. Every 100ms delay in page load time is estimated to reduce conversions by 1%. For a fintech site processing thousands of loan applications monthly, even small speed improvements translate to meaningful revenue increases.
The largest contributors to slow pages on Indian fintech sites are: unoptimized images (the #1 culprit), render-blocking JavaScript (especially third-party scripts like chat widgets, analytics tags, and ad trackers), server response time (aim for TTFB under 600ms), and large CSS files. Use PageSpeed Insights or Google Search Console's Core Web Vitals report to identify which specific issues affect your pages.
| Technical Issue | Impact | Fix | Priority |
|---|---|---|---|
| Blocked by robots.txt | Pages not indexed | Review and correct robots.txt rules | Critical |
| Noindex tags on live pages | Pages not indexed | Audit meta robots tags, remove from important pages | Critical |
| Slow LCP (>4s) | Rankings + conversions | Compress images, defer JS, use CDN | High |
| Broken internal links (404s) | Crawl efficiency, UX | Redirect or fix broken links | High |
| Missing HTTPS | Security signal, slight ranking factor | Install SSL certificate, redirect HTTP to HTTPS | High |
| Duplicate content | Diluted rankings | Canonical tags, consolidate content | Medium |
| No XML sitemap | Slower discovery of new pages | Generate and submit to GSC | Medium |
| JS-rendered critical content | Content not indexed | Server-side render key content | Medium-High |
HTTPS and Security
HTTPS (the secure version of HTTP, using SSL/TLS encryption) has been a confirmed Google ranking signal since 2014. More practically, browsers like Chrome now flag HTTP sites as "Not Secure" — which destroys user trust and click-through rates, especially critical for financial sites where users are entering sensitive data. If you're still on HTTP in 2026, this is your first technical SEO fix, full stop.
Beyond the basics, HTTP/2 (and increasingly HTTP/3) provides significant performance benefits over HTTP/1.1 — multiplexed requests mean browsers can load multiple resources in parallel rather than sequentially. Most modern hosting providers enable HTTP/2 automatically with HTTPS. Verify yours is enabled using a tool like KeyCDN's HTTP/2 test.
Mobile-First Indexing
Since 2023, Google uses mobile-first indexing for all websites — meaning it crawls and indexes the mobile version of your site, not the desktop version. If your mobile pages have less content, missing structured data, or slower performance than desktop, your rankings will reflect the mobile version's quality — even for desktop searches.
This particularly affects Indian fintech sites that historically served simplified mobile pages with truncated content. If your "personal loan interest rate" page shows a full rate table on desktop but collapses it behind a "show more" button on mobile (and the collapsed content isn't in the initial HTML), Googlebot sees the truncated version. Ensure all critical content is in the initial HTML response on mobile — not hidden behind JavaScript interactions that Googlebot may not execute.
- Technical SEO is the foundation — a single misconfiguration can de-index thousands of pages overnight, wiping out months of content work.
- Robots.txt controls which pages Googlebot can crawl — always test changes using GSC's URL Inspection before deploying to production.
- XML sitemaps speed up discovery of new content — submit them via Google Search Console and keep them dynamically updated.
- Google uses mobile-first indexing — your mobile pages determine your rankings. Ensure content parity between mobile and desktop.
- HTTPS is now table stakes for any site, especially financial services where user trust in data security is paramount.