WAGGENER EDSTROM is a PR firm which markets itself as helping clients “unleash [their] brand’s full potential.” They help firms “leverage” the “influence architecture” of the internet to “supercharge” their “power spunk” and achieve “MONSTER! satisfaction” – or some such jargon. Anyway, they’ve just released a report called “The Influence Index” (a.k.a the “Brussels Blogger Study 2010″) about the state of the Euroblogosphere – and it’s turning some heads… but perhaps not for the right reasons. For one thing – Jon Worth (who Waggener Edstrom considers the 5th most influential EU blogger) had to invite himself to the conference at which the report was being presented. Jon gave a fairly nonplussed account of the conference from Twitter.
Then, last night, the ghost of a famous euroblogger was controlling the @bloggingportal twitter account and completely demolished Waggener Edstrom’s report. I’ll reproduce Bloggingportal’s findings here – with full credit to the anonymous twitter account user for the research. Essentially, the problem is that the report used somewhat arbitrary selection criteria and then failed to keep to that criteria when choosing blogs. Ultimately, they also more-or-less fail in their overall objective – which is to “identify the key voices in the Brussels influencer landscape.” They do identify many key voices, but they leave out as many influential blogs and their so-called “ranking of influence” is a waste of time.
- Are they current? Bloggers must have posted in the last two weeks at least. Some interesting blogs were not ranked because activity had trailed off, calling their sustainability into question. Actually, this seems like a reasonable requirement. However, the #3 ranked blog (The Digger) only has four posts in 2010. In fact: it only has 12 posts in total. That’s right – the #3 blog in the euroblogosphere has only posted 12 times (those must be some amazing posts). The #1 speciality EU blog has no posts for the last 2 months. And two of the highest-ranking blogs – Julien Frisch and Charlemagne – have closed down (how’s that for “calling their sustainability into question”? Though, granted, the loss of Charlemagne is only temporary).
- Are they frequent? Bloggers must be posting at least twice a month. Building up a following and being influential requires effort and application so that people can be sure to hear your point of view. See above. In addition, the #6 specialty blog has posted nothing in July (which we’re now halfway through), twice in June, nothing in May and twice in April. Did Waggener Edstrom mean to say they must post at least twice every two months?
- Are they writing on EU matters? We selected only blogs that cover (or mainly cover) EU affairs. Many blogs with occasional references to EU affairs (e.g. Wall Street Journal blogs) were not included in the survey. Firstly – what’s wrong with the Wall Street Journal’s EU blog? It’s devoted to the eurozone and EU politics. Secondly – the #6 “speciality” blog only mentions the EU in 3 of the last 10 posts, the #4 “speciality” blog is technically about the ECHR (not the EU), and the #30 “Brussels Influencer” made no reference to EU matters in the last 15 posts.
- Are they EU-based? We excluded blogs based in the US where additional visitors could skew results. Aaah, so THATS what’s wrong with the Wall Street Journal blog. This seems reasonable, although it’s a bit odd that they’ve excluded the WSJ’s Real Time Brussels blog (which is Brussels-based) just because its parent newspaper is US-based. If they really wanted to exclude additional US visitors, they should have looked at non-English blogs.
- Are they writing in English? To enable comparison with a similar study carried out by Waggener Edstrom in the US in 2009, we selected English language blogs only. This is an absolutely ridiculous criteria. The European Union has 23 official languages, and Waggener Edstrom thinks an accurate reading of the blogosphere can be done in just one language? It might have been more reasonable (though still shoddy) to have looked at blogs in the three working languages (English, French and German). The excuse is that they wanted to make an accurate comparison with the US blogosphere… and yet by excluding all the non-English blogs, they are providing a fundamentally flawed representation of the European blogosphere – so any comparison with the US is rubbish.
So, Waggener Edstrom has a broken set of selection criteria which they aren’t even able to apply properly. However, their methodology is equally flawed – which might explain how they managed to reach such bizaare results. Their methodology is based on:
- Content Focus & Relevance – i.e. the frequency and depth of mentions [of the EU]. Measured by content analysis. This is a no-brainer (especially as measuring EU blogs is the entire point of the study). No further comment.
- Target Audience – i.e. the importance/relevance to the target audience. Measured by content analysis. This is a bit fluffy. How do you measure the importance/relevance to the target audience through content analysis? Wouldn’t it be better to, y’know, ask the target audience? Get in touch with some commenters or carry out surveys? Form focus groups and ask them to read specific blogs for a month, then conduct interviews? If this isn’t actionable given the amount of resources available, then don’t include it.
- Reach – i.e. the unique visitors, circulation, site rank, followers. Measured with Cision, Alexa, Quantcast and Twitter. There are all sorts of issues with sites that measure the readership of websites. For example, your readers all need the Alexa toolbar installed before they are counted by Alexa. How many readers a blog has is also often not so important as who the readers are. The euroblogosphere has a piddling audience-size compared to other blogospheres (for various reasons) but it can still reach influential people.
- Buzz / Pass Along – i.e. the blog links, blog rank, aggregation and popularity. Measured with Digg, Techmeme, Technorati and Blog Pulse. Now this is problematic. Twitter would be the perfect way to measure buzz in the euroblogosphere, but it was explicitly excluded from the study. Digg and Techmeme are great for measuring buzz around technology (and possibly US politics) but lousy for anything else. Nobody bothers registering with Technorati anymore, and even fewer people use Blog Pulse.
Damningly, the report makes no mention of actually interviewing eurobloggers and asking them their take on the state of the blogosphere. It also doesn’t make use of any of the previous research done into the euroblogosphere (here, here and here, for example). It had flawed selection criteria (which it then failed to implement correctly) and a flawed methodology.
The problem is that Waggener Edstrom thought they could approach the European blogosphere and use their carefully-developed commercial tools to understand it (obviously without the necessary language-skills to even utilise their own tools fully). But the EU blogosphere is in fact so small that most of us now know each other and have often met in real life. If Waggener Edstrom wanted to do a proper analysis, they should have interviewed some actual eurobloggers rather than tried to understand a small community of bloggers from the outside.
Agreed, just a few additional observations:
The rationale behind the selection of the blogs (even in English) is not well documented. One could even come to the conclusion that it was a random sample. Several important blogs are missing. (including blogs from Bxl consultancies…)
Another example are the “European Voice blogs” I am not aware of any such blog although on the EV website you can find this one – which is really not a blog!
And regarding methodology: It is not enough to just make some statements about “relevance” and “buzz” without any looking at definitions and explaining the indicators. There is no mention about a time frame: Did they monitor blogs for 4 weeks or 12 months? The numbers in the different influence table look very ‘scientific’ – but it seems that they only reflect the interpretation of the researcher and are not based on a more elaborated methodology. And even if it is more sophisticated, the report does not mention the exact calculations (or the indicators for the content analysis). The lack of any interviews is indeed striking – as this could have provided a useful reality check. More generally, it is questionable whether metrics that were developed in a US context can be applied to the EU context. In fact the EU (and the Brussels bubble) as such is very different from the US (and the Washington scene) so it is a rather lazy exercise to just use the US template for the EU…
Another rather curious #fail: It is quite an achievement to publish a blogger study without including any links!
It’s just a shame the only real criticism the report is getting is coming from the blogosphere (i.e. from the actual thing they’re supposed to be measuring). The only mainstream coverage I’ve read has been rather positive (with slight criticism for being English-only).
Pingback: Mathew Lowry’s Tagsmanian Devil » Blog Archive » That euroblogger influence survey (updated)
Just updated my own post on the subject – you’ve done an awesome demolition job here!
Were you actually at the conference, or is this analysis based on their report? I was sent to the wrong address, which was a shame, as I really wanted to ask them about their methodology.
I think we do need something like this. The hope remains that W-E get it right next time, or someone else shows them how to do it.
I’ve just left a response on your blog.
Pingback: Better Know A Euroblog | europaeum
Pingback: EuRoman » Blog Archive » Brussels Blogger Study 2010 just out: could do better, outside Brussels
Interesting points, you triggered a real debate!
Still, I do see some value in that survey, you do the next one?
I just contributed some more feed-back here:
http://euroman.blogactiv.eu/2010/07/15/brussels-blogger-study-2010-just-out-could-do-better-outside-brussels/
(and added a link to you at Matthew’s suggestion, thanks)
Christophe
Thanks for your interest. A lot of good points – the scale and scope of the project could indeed be increased and that’s exactly the type of work we do for our clients – our report was designed to get people thinking and talking. I would like to specifically point out that our study provides a snapshot of the Brussels blogosphere from earlier this year due to the timings between the analysis and the event so, as you rightly point out, influence is constantly changing and needs to be evaluated accordingly.
On the language front, we agree totally that our next survey of EU blogging just has to take account of linguistic diversity. Clearly more languages would have been desirable but we were also concerned about finding a reasonable survey sample size, and felt that casting our net across all languages would have forced us to make some other arbitrary choices about who to include and rank.
We’ll continue to track the response to the survey. Many people have commented positively on the underlying idea of trying to measure and rank for influence, which we’re glad about. We think trying to measure and track influence is an important and worthwhile endeavour. We’re certainly not suggesting that our approach is anything like a final methodology, especially not in a Brussels setting. But we’re delighted that the discussion is going about influence. No one organization can develop the ultimate ranking methodology, but collectively we can do a lot more. I would hope that future iterations of this could be done with extensive consultation not just with the blogging community but also with companies, policymakers and the rest.
For a little more background on what we were trying to achieve, check out this interview:
http://stuartlangridge.blogactiv.eu/2010/07/14/which-is-the-most-influential-eu-policy-blog/
John, I appreciate you taking the time to reply to what is a very hostile review. Your report has sparked off a discussion across multiple blogs – Christophe has provided a round-up of some views in the post he linked to above and the last five front-page posts on Bloggingportal.eu right now are talking about your report. If you wanted to spark a debate, you have succeeded – many of the top English-language euroblogs you mentioned are now discussing your report.
I thought your research might have been done earlier in the year, and I accept that the euroblogosphere changes often… but in order for The Digger to have achieved the #3 spot your research must have taken place in 2009 or early 2010, when most of the 12 posts made by that blog were written. It’s now July – which is a long time to collate data and write a report if you want the research to remain relevant.
I wish you the best of luck with further reports, because there’s a definite place for more research into how blogs might influence politics in the EU. I would suggest that a lot of the most interesting results are to be found in examining the human relationships built between EU bloggers and politicians, lobbyists, NGOs, etc. – and sending out e-mail questionnaires next time to supplement your online research might yield results.
One last thing on buzz – the number one generator of buzz in the EU sphere is Twitter. I know you excluded it from your study (and I understand it would have taken extra resources to study) but it’s really how EU bloggers pass on buzz.
Ok, that’s enough criticism from me. If I was harsh, it’s because I think there is a place for your report, but I didn’t agree with your results. Thanks for your comment (on a hostile blog), it’s appreciated.
Good critique.
I feel the study’s heart was in the right place, but importing a methodology used in the US and the comparisons with the US blogging scene as if this something that the Euroblogosphere should be aspired to become like may also have added to the distortionary effect.
While in the US the Washington world is probably exciting enough to fully occupy bloggers, most EU blogs I read seem to also have interest in other things – whether that’s Jon Worth’s sportsblog or Joe Litobarski’s musings from Ethiopia.
I’m an occasional euroblogger, who, through a combination of not-covering-some-things-because-I-value-my-job and blogging on things other than the EU (primarily parenting, feminism, local issues and faith), is never going to make it up the rankings.
While I understand that my own blog’s too random to fit the primarily EU-focussed criteria, I’m a bit surprised that none of the blogs of the EU girl geeks appear even to have been in consideration: where was Europasionaria? Euonym? Lino the Rhino? Or did I just miss the longlist of blogs that were considered?
No EU girl geeks, no non-English blogs and no eurosceptic English blogs (the EU referendum blog by Richard “slaughter them in their beds” North gets many, many more hits than most pro-EU blogs).
As Europasionaria argued recently, we need to blog more about sports and Ethiopia (and other things) and blog less about euroblogging.
Glad to see the conversation developing in this way. We can all agree that the study could be a lot better: totally agree about the missing EU girlgeeks, and Jon already mentioned a few other anomalies.
But these are early days. You should have seen the first few blogs set up on Blogactiv – they were d-i-r-e! But I was happy to get them. The quality has improved since then. The same thing will happen in tracking, analysing, sorting, aggregating and EU social media. And all of that will grow the EU online public space.
But it would be interesting to see some competition in this area, so if people think they can do better than W-E, then they should.
Definitely agree with you about more competition – I would love to see some more research done into the euroblogosphere – and especially some research which suggests ways we can grow it. However, I wonder if we’ve been a bit over-indulgent with this recent report. It’s nice to be noticed, but I feel like I’ve said everything I want to say about euroblogging now – and I want to write more about Europe and the EU and less about blogging.
But it won’t be long until I’m back on the topic of euroblogging again, I’m sure.
Pingback: That Manifesto for a New Europe which Nobody Read | Eurogoblin.eu
Pingback: bloggingportal.eu Blog & Support » Blog Archive » The Week in Bloggingportal: My Euroblog is Better Than Yours!
Pingback: Euroblog Round-Up: My Euroblog is Better Than Yours! | Eurogoblin.eu
Pingback: Blogactiv Director » Blog Archive » Waggener Edstrom - Where Are You?
To all of you that agree regarding the critics towards “Brussels Bloggers Study 2010″ I have a question;
* Which well written blogs would you advice me to read if I want to keep myself updated regarding the politics of the European Union?
It really depends which languages you speak, what your political preferences are, what areas of the EU you’re interested in and what sort of writing style you enjoy. The list in the Brussels Bloggers Study 2010 is certainly a group of A-grade blogs (it’s more the way they’re listed in terms of “influence” that has come under the most criticism). Otherwise, you could try exploring Bloggingportal (where I’m an editor) to discover new EU blogs (it lists over 600).