Performance Zone is brought to you in partnership with:

Chris Kelly is a software craftsman and open source evangelist working to maximize developer happiness. With experience that has touched all layers of the web application stack, Chris speaks, blogs and opines on Ruby, DevOps, engineering culture, open source and soccer. He is currently the Developer/Evangelist at New Relic. Chris is a DZone MVB and is not an employee of DZone and has posted 3 posts at DZone. You can read more from them at their website. View Full User Profile

Real-Users, Performance, and What Matters to You

07.12.2012
| 4115 views |
  • submit to reddit
Web Operations and Performance Summer Camp, better known as the Velocity Conference, has come and gone. This year’s theme of Building a Faster and Stronger Web brought geeks from the #devops and #webperf communities together to discuss how to push the web forward given the continued growth into cloud and mobile. The two-day conference schedule was jam-packed with passionate, brilliant people all driving the web into the future. We’re very lucky to have such motivation and desire within our community!

Web performance is a growing field within the greater operations community. This year it gained its own WebPerfDays – modeled after the highly regarded DevOpsDays – both of which followed the main Velocity conference. You couldn’t attend a #webperf session without some discussion about the evolution of performance benchmarking. In the “early” days, web performance was measured using clean-room markers such as synthetic tests, time-to-first-byte and onLoad events. If the sessions are a sign of things to come, performance benchmarking will soon focus on the user experience. This isn’t to say that the previous benchmarks were wrong. Instead it shows we’re gaining a broader perspective on web performance.

Mike Brittain, Director of Engineering at Etsy, reminded us to focus on our end user experience. At Etsy, features such as ads, login systems and social sharing can fail to load without degrading the user’s primary experience. The only features that must load (and can therefore block loading), are the product itself and the “Add to Cart” button. Everything else is tertiary. In this way, Etsy has made sure that the features that are core to its user’s experience are also core to its overall business.

Patrick Meenan, of Google and webpagetest.org, took another look at the user experience argument. In his talk, Patrick discussed how an arbitrary marker, such as onLoad, is insufficient in determining performance by showing a filmstrip view of Amazon and Twitter side-by-side. As you can see below, users have an opportunity to engage with Amazon much sooner than with Twitter. Even though the onLoad event is the same on both sites, the user experience is drastically different.

In a standing room only panel discussion on web performance monitoring tools, our own Lew Cirne joined web performance gurus Patrick Meenan, Joshua Bixby (of StrangeLoop), Marty Kagan (of Cedexis), and Patrick Lightbody (of Neustar) for a final look at the real user versus synthetic monitoring theme. The key takeaway is that there is room and need for both. Synthetic monitoring cannot replicate how users are actually using our applications; what’s important is how our users really use our applications, not how we think they use them. On the other hand, most sites do not generate enough data with real user monitoring to have a statistically relevant sample and so any conclusions are usually accompanied by an asterisk. As with most things, there is no single right or wrong option but rather some combination of both.

And our own Director of Product Marketing, Bill Hodak, gave a quick overview of New Relic and our Application Speed Index feature as part of O’Reilly’s Best of Velocity from the show floor.

In the end, we can all agree that what is critical is to find a metric that is most important to you. For Twitter, it is time-to-first-tweet that drives their performance focus. What metric is most important to your business?

Can’t wait to see you at Velocity 2013!

Published at DZone with permission of Chris Kelly, author and DZone MVB. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)