How to do comparative media analysis on the web: Start small.
I’ve always thought that in order to understand a network, like the web, better, we should be able to understand as much of the objects and actors that the network itself is made of, as possible. Whilst finishing my BA in Communication and Media Studies, I wanted to write my thesis about how/whether the structure of a comment space effects the conversation that is being had on the comment space. As I delved a little bit more into the theory, I realized that I’d rather initially write a thesis on the theoretical aspects of this question -more generally, about how to employ Actor-Network Theory to web research and abandoned the topic for a while.
During the Digital Methods Summer School in 2010, we had the opportunity to pitch our ideas to the whole group, and if enough people joined, then we could work on them for a week. So, during the last week, I talked about this idea and two other participants of the school agreed to look more into this, whilst employing digital methods to the question. We changed the question a little bit, because -as I will lay out the reason later- we realized, that this question is basically unsolvable. Our research questions by this point were:
1. What types of commenting systems can be distinguished within the most distributed comment spaces?
2. What levels of user participation do these systems enable?
3. How do the selected websites rank in alternative comment spaces (Facebook, Twitter, Digg, Reddit…etc.)?
4. What is the relation, if any, between the alternative comment spaces and the selected websites?
First off, I really wanted to have a categorization of various commenting systems according to the typologies of comment sorting, viewing, sharing, replying, linking, recommendation, and also registration requirements. Of course, in order to do this we’d have to analyze a substantial amount of popular websites’ comment spaces. Everybody who has spent time reading articles online knows that the more controversial and issue, the more commenting and engaging with the issue can be recognized. We chose ‘ground zero mosque’ as our controversial issue and scraped Google for this query. After retrieving the first 1000 results, we only kept the links that had comments and inlinks from at least two social media platforms, in order to be able to answer our question about distributed comment spaces.
After ordering the platforms based on how participatory the comment spaces that can be found within them were (with the help of using meta-categories) we also looked at how these articles were represented on various social media platforms. We found that there isn’t a correlation between how participatory the comment space on the original platform is and how much engagement was going on with the issue outside the platform, which might seem obvious, but wasn’t my initial intuition, so I was happy to find out something that contradicted what I assumed. (Of course, we didn’t look into whether there is an emigration phenomena (to social media platforms) going on with regards to news sites that do not allow commenting.)
Going back to the original question about whether there is a relationship between the technicalities of a comment space and the kind of conversation that it enables: e.g. I assumed that the comment layout of Reddit made it easier for users to carry out long conversations, because the reply function has unlimited iteration (the conversation doesn’t have to end at one point) and it is indented (easier to follow). I also assumed, that the voting system (Facebook has Like, Reddit has upvotes and downvotes) has an effect on the developing of the “echo chamber” (or preventing it). This question still interests me, but there are various reasons why this could be a question that I am unable to answer without my findings getting manipulated by platform-based and algorithmic bias.
Problems:
1.) Platform-based bias: You are conversing using your real name on Facebook/Your replies on Twitter come from a context, within which you already have a “portfolio” on the platform itself, it isn’t just a comment, it is the continuation of your Twitter identity, which in this case is very explicit. People comment in a platform-aware way, conversing on e.g. New York Times cannot be the same as commenting on cracked.com (although, I have not researched this, and I might be wrong).
Solution: Build approx. 50 different types of comment spaces, find a representative sample of internet users and get them to engage with a sample of issues on these different spaces, in the end, we’d have a clean set of data to analyze and compare, that doesn’t take into account the various qualitative aspects of the platform itself. (Of course, if it is “clean data”, then it is very much separated from the real world, which might be an even bigger problem.)
2.) Algorithmic-bias: This is actually what I am trying to prevent, I would like to be able to categorize comment spaces and ways of engagement based on pure technicalities, just so we can normalize various platforms when researching them: If we know what kind of an algorithmic bias is imposed onto a conversation going on Twitter vs. on a Facebook Page, we can be able to research the conversations on these two platforms and actually compare them, if we are able to take this bias into account.
3.) Research methodology: Other problems include the vast amount of content analysis a research like this would necessitate, which I do not have much faith in as a primary research tool.
The reason I am very interested in these types of questions and believe in the importance of analyzing various different digital/digitized objects of the web is exactly because of these two biases that I mentioned above. There are lots of data-driven new media (especially Twitter) research, that employs network scientific methods and think of the web as a network with edges and nodes and come up with a network that shows the flow of the information or the hierarchy of credibility within the network…etc. and try to answer open questions of communication and social science research. But the eternal question of data-driven, network scientific internet research remains: Are our findings about these theoretical questions or solely about the platform we are basing our research on? The only way to answer that would be to do a comparative media research, but how can we find the common ground over which we can look at an issue on Facebook and on Twitter? How do you normalize these two platforms to be able to do comparative media research?
What I am suggesting is starting small: Let’s look at the objects and try to understand the way they work and the things they privilege and limit. Then, one would have to look at the interactions between these objects, and only after this can we have a network that includes a substantial amount of actors over the web (and not only over one single platform).