« Preparing for the upcoming April 2008 Flash Player 9 Security Update | Main
August 28, 2008
Two! Four! Six! Eight! Numbers we appreciate!
I write every once in a while about Flash Player statistics, which I hope is interesting and useful for you. As product managers for Flash Player, Justin Everett-Church and I are responsible for managing and interpreting the data related to Flash Player. That data includes the penetration study, as well as player download and installation metrics. So, forgive me for bringing up old news, but I was out on vacation when the Firefox 3 Download Day happened. I was (quite happily) checked out enough to not really hear anything about it until I got back in the office this month. I understand a few Adobe folks blogged about how our Flash Player numbers were also world-record worthy, and a lot of different download numbers often show up in blogs and press without any context of time period. Is the number of daily downloads a useful metric? What do these numbers really say?
Don’t get me wrong -- I love data, and download numbers ARE interesting. But, as with any data, you need to figure out what the numbers tell you and why they are important. Adobe tends to talk about the install rate of Flash Player quite often, although it may not be the right number for people to focus on since it doesn’t give you context, and you have to take our word for it. For Firefox, the download number is interesting because they wanted to beat their numbers for the previous release, and to generate buzz for the current release by submitting it for a world record -- which was verified by a third party.
It’s interesting to note that for Adobe, the number that is quoted is an “install” and not a “download” number. We’ve never said how many downloads happen a day because it is a ridiculously large number AND we know that it’s not that useful metric because those successful downloads don’t all turn into successful installations. In July 2008, successful downloads averaged about 33 million per day, and successful installs averaged around 18 million per day. That seems like a big drop, but consider that ActiveX was about 80% of our installs that month and when you visit a page that triggers the ActiveX install experience the installer is downloaded to the machine before the security warning dialog appears. The user might say “no thanks” to the security warning dialog, and refreshing the page or visiting another page that requires a newer version may download the installer again.
How do we monitor Flash Player download and installation numbers?
We have an internal dashboard application to track the general “health” of Flash Player downloads and installs, and it can chart the data daily, weekly, monthly, etc. It uses XML feeds of our server log data from Akamai, our current CDN. The dashboard is useful in helping us 1) to understand our traffic so we can try to optimize things like install success rate, and catch problems with our installers or CDN delivery in a matter of days, and 2) get a sense of where our penetration might be in the penetration study in the next wave. The penetration study is only run once a quarter, and it’s hard to wait three months to know where we’re at.
For “Player downloads” we count the attempted and successful downloads for all the player installers we post to the CDN (based on the related HTTP status codes), such as “swflash.cab” and “install_flash_player.exe” (note these numbers aren’t unique.) We also count the number of installs, which is the sum of requests for a small text file that a newly installed player requests the first time it is launched in the browser. That number is used as an estimate – we’ve had releases where the text file request wasn’t implemented or working on certain platforms, and since it is only requested once there are a number of reasons why it might not make it all the way to the server. The install number wasn’t ever intended to be a marketing point, although it was an exciting number for us to talk about when we realized that our install average was 8 million a day shortly after Flash Player 9 first launched. The daily average has gone up since then, most likely due to increasing penetration of broadband and improvements at the CDN. And it can spike depending on whether there is a hot new site that is sweeping the web, or dip if we’ve got a CDN or installer problem.
The point is, I know it can be confusing to hear about numbers like 8, 12, 14 or 25 million daily installs when there isn’t any context in which to interpret the meaning. We don’t yet have an internal “standard” for the install number that we are quoting as a company, such as “average daily installs for the past month” or “average daily installs since the last release”, etc. Not that it really matters. The intent behind stating these VERY BIG numbers is to say something about demand. That is a big reason why we have the penetration study, and have been tracking it since Flash Player 3. I’ve talked in the past about the general aspects of the penetration data we post, and how it may or may not apply when you get down to your particular audience. As with any statistics, you should understand the methodology behind the numbers. But the good thing about that number is it is something you or another third party like Forrester, can independently test or verify -- which makes it the more interesting and important number for Flash Player.
Posted by ehuang at August 28, 2008 02:00 PM
Trackback Pings
TrackBack URL for this entry:
http://weblogs.macromedia.com/mtadmin/mt-tb.cgi/9410
Comments
Interesting read Emmy! :)
I'm actually glad someone with a bit of ownership stepped forward and owned this problem from Adobe, so kudos for that! :)
Question, I've seen your staffers throw the "world-wide" message around whenever we're in the room etc. What we don't see is what this number looks like between Consumer (i.e. Web) and behind the firewall. In that the Penetration Study doesn't seem to clarify the behind the firewall for instance?
So, I guess, the question is out of the Fortune 1000 companies, what would be the penetration here and could you clarify the penetration studies methodology in how this is applied?
thanks! :)
-
Scott Barnes
Microsoft.
Posted by: Scott Barnes at August 28, 2008 08:15 PM
Hi Scott,
Yes, that is a question that comes up often. From the methodology, "Respondents must be Adults 18+ and have access to an Internet connected computer.
The majority of respondents answer from their home computer; however, responses from respondents answering using a work computer are not excluded. A question is included in the survey to track whether the respondent is answering on a home computer or a work computer."
What that means is -- while we do know whether someone answered from their work or home computer because the question was asked, the study/sampling isn't weighted in a way where we could really say "x% of work users have Flash Player." We don't ask them if they are at one of the Fortune 1000 companies, and even if we did it wouldn't be valid to say a percentage based on the way the sampling is done.
So the answer to your direct question is that we don't specifically test the penetration within Fortune 1000 companies in the Millward Brown study. Maybe the Fortune 1000 wants to volunteer that information? ;-). But, you could check the Forrester report to see what their methodology was for enterprises. They surveyed 50K enterprise users. From the article I linked to above:
"Forrester's study examined the browser as well as the desktop environments of the 50,000 users, spread out among 2,300 companies."
"Both Flash and Java were nearly ubiquitous. Flash Player version 9 was on 97 per cent of desktops, while Java was on 99.9 per cent of them."
The report is here, but does cost money: http://www.forrester.com/Research/Document/Excerpt/0,7211,45898,00.html
cheers,
Emmy
Posted by: emmy at August 28, 2008 10:02 PM
For what it's worth on corporate support levels, back when Chris Swenson was at Macromedia and processing the NPD/MediaMetrix consumer audits, there were a few quarters when corporate accounts were separated from home accounts.
The overall penetration rates were essentially the same for business and home, even though corporate decisions are usually "lumpier" than decentralized individual decisions.
Whether a particular worker's computer supports rich-media depends on their particular intranet, and Chris's testing could have benefited from corroboration, but overall, the penetration rate in enterprise has seemed to match general consumer norms, as that more recent Forrester report confirms.
jd/adobe
Posted by: John Dowdell at August 29, 2008 09:26 AM