HTTP Archive uses Wappalyzer to track the state of the web

The HTTP Archive tracks how the web is built. It provides historical data to quantitatively illustrate how the web is evolving over time. The project is open-source and backed by technology giants such as Google, Mozilla and Akamai.

The initiative is part of the Internet Archive, a nonprofit organisation that provides free public access to digitised materials, including millions of books, software applications and websites. Its web archive, the Wayback Machine, contains hundreds of billions of historical web pages.

The HTTP Archive evaluates the composition of millions of web pages on a monthly basis and makes its terabytes of metadata available for analysis on BigQuery, a petabyte-scale data warehouse by Google.

In one such example, Paul Calvano, web performance architect at Akamai, demonstrates how the dataset can be leveraged to analyse CPU times across JavaScript frameworks with the help of Wappalyzer.

Web Almanac

Once a year, the HTTP Archive produces a comprehensive 'state of the web' report called Web Almanac, backed by real data and trusted web experts. Its many chapters span aspects of page content, user experience, publishing, and distribution. While the Web Almanac is primarily the domain of developers and web architects, it's full of helpful stats for content managers, marketers and publishers. It's an invaluable resource to understanding how the web is trending.

Notable insights in 2019 include:

  • jQuery is used on 85% of web pages
  • 10% of pages are on an ecommerce platform
  • 40% of pages are powered by a CMS
  • Only 20% of sites serve HTML content via a CDN
  • At the 90th percentile, 91% of bytes on pages are from media

In addition to tools such as Google's Lighthouse and Chrome UX Report, The HTTP Archive adopted Wappalyzer to segment reports and help break down the aggregate analysis of how the web is doing by various technologies. The Web Almanac project uses Wappalyzer extensively as a way to compare the performance of all technologies within a category, such as CMS and ecommerce.

The Web Almanac is the brainchild of Rick Viscomi, a software engineer and web performance evangelist at Google, who noted that Wappalyzer is well-aligned with their principles of web transparency because of its open and community-driven approach to software development.

WebPageTest

WebPageTest, created by Catchpoint software engineer Patrick Meenan, is a prominent web performance testing tool and the backbone of HTTP Archive. It makes use of Wappalyzer's profiling methods to analyse web pages and create security and performance reports.

Conclusion

Successful societies and institutions recognise the need to record their history, providing a way to review the past, find explanations for current behavior, and spot emerging trends. The HTTP Archive provides a permanent record of how digitised content is constructed and served.

Wappalyzer is proud to play a part and continues to work directly with core maintainers of the HTTP Archive project to add and improve technology fingerprints and methods of inspection.

Website profiling
Find out what websites are built with.
Lead generation
Find prospects by the technologies they use.
Market research
Compare market shares and technology trends.
Competitor analysis
Discover who uses competitors' software.
Data enrichment
Technology, company and contact information.
Custom reports
Create lists of websites and contacts.
Website monitoring
Monitor website technology changes.
Browser extension
See the technologies on websites you visit.
CRM integration
See the technologies of your leads.
Email verification
Improve delivery and mailing list quality.
API access
Instant and real-time technology lookups.
Security recon
Reveal web technologies and version numbers.
Apps

Wappalyzer works with the tools you use every day.

Chrome

See the technologies of websites you visit in your browser.

Firefox

See the technologies of websites you visit in your browser.

Edge

See the technologies of websites you visit in your browser.

Safari

See the technologies of websites you visit in your browser.

HubSpot

See the technology stacks of your leads in your CRM.

Pipedrive

See the technology stacks of your leads in your CRM.

Zapier

Connect Wappalyzer to the apps you use, no code required.

Integromat

Connect Wappalyzer to the apps you use, no code required.

Pabbly

Automated workflows and email marketing.

Gmail

See the technology stacks of your contacts in Gmail.

Android

Wappalyzer in your pocket.

Wappalyzer is trusted by thousands of professionals world-wide

Wappalyzer has proven to be a great tool to help us break down the aggregate analysis of how the web is doing by various technologies. Ilya Grigorik
Principal Engineer at Shopify
These days you need advanced marketing tools to stand out from the competition. Wappalyzer help us do just that. Thomas Alibert
Growth Engineer at PayFit
Wappalyzer is an integral part of our sales process, enabling us to optimise lead segmentation at scale. It’s a total game changer for our organisation. Roman Schweiger
Head of Business Development at Boomerank
Wappalyzer has been such a useful part of the HTTP Archive dataset. It's enabled us to slice the data in new ways and discover more interesting insights about the state of the web. Rick Viscomi
Senior DevRel Engineer at Google
Wappalyzer is helping our sales teams to understand prospects better and faster by having a clear view on their tech stack. Rabin Nuchtabek
Chief Growth Engineer at Skedify
Wappalyzer has been hugely valuable to slice performance data by the various detected applications at WebPageTest.org. Patrick Meenan
Engineering Fellow at Catchpoint

Subscribe to receive occasional product updates.