Solving Filter Failure
Filed under: 
Publishers are increasingly concerned about “information overload,” and some believe that technology has made this worse. In an Economist.com debate, Richard Szafranski argued that technology has created “over-choice,” which he described as a “human response to alternatives and variations so numerous, so potentially satisfying and so complex that humans can no longer decide easily.” Our time is limited and the more choices we have, the more time it takes to choose.
However, Clay Shirky claims that information overload has been a problem long before the digital age, as anyone has experienced entering a library or bookshop. Ever since the amount of available books exceeded a person’s ability to read them, the central problem has been filter failure. “We had a set of filters that we were used to, but are now broken,” Shirky points out. The media organizations that help solve filter failure by making information more relevant will control the new decentralized online distribution channels. Independent media has more power to solve this problem by sharing data and working together.
Metadata
Top-down ways of categorizing content for users based on a library/bookstore metaphor have disappeared online. Metadata has enabled a more decentralized system for users to find content and content to find users.
Metadata is simply data about data and comes from hyperlink relationships, contextual content, tagging, and provenance (the when, where, and how information originated), among other inputs. It can reveal users’ intentions and enable computers to infer meaning about personal preferences, trustworthiness and reputation among many other things. Metadata is the secret to help users find information and make it more relevant and useful to them.
At first, search relevancy was mostly derived from the text on the page, a user’s search term and a website’s link information. Metadata was an afterthought for publishers; Search took care of itself. Today, publishers use Search Engine Optimization (SEO) to leverage contextual and link metadata. However, filtering is becoming more complex and important. It includes social connections, location and user behavior such as search history and movement. If independent publishers share metadata in more sophisticated ways, they can make their content relevant and useful across multiple platforms.
Metadata can also increase the value of data as news content. In fact, a media outlet’s “primary asset will be the completeness and reliability of its records” [metadata] rather than news stories according to Dan Conover, former city editor of the Post and Courier and 2005 Journalist of the Year in South Carolina. For example, a journalist who covers a house fire would collect additional data that may not make the final story, explained Conover on the group blog Xark. “How many alarms? Official cause? Forest Fire (y/n)? Official damage estimate? Addresses of other properties damaged by the fire? [...] names of victims.” This metadata would feed into a relational database, which allows it to be compared to other data, used for future stories, and sold. Who might pay for such a data set? “Well, insurance companies, for starters, but perhaps also attorneys, the Red Cross, real estate agencies, marketing companies, private detectives, specific vendors, etc.,” wrote Conover. Similar scenarios could apply to a wide variety of local and national news.
Due to their journalistic training, reporters could collect highly trustworthy metadata. Most online users hand-tag for personal reference or to game the system for more views, which makes user-defined metadata less reliable. As the reliability of metadata grows, so does its value. If organizations cooperate to collect data in similar ways and share it with each other, they will make
information more useful and valuable over time.
News will likely become even more fragmented and granular than it is now. People are already pulling together many tiny pieces of information from lots of sources to form a picture of what is going on. As content further proliferates and converges across devices, users will increasingly turn to curators, whether it’s friends, friends of friends, trusted outsiders, publishers, automated filters, or more likely, a combination of them all. The media organizations that can help solve filter failure—equal to their expertise in content production—will most likely succeed in the future.
To see more analysis on solving filter failure, download Vol. 2 of The Big Thaw.
[...] this do for a media producer? Let’s play it out a bit. First, the media producer is solving “filter failure.”, a big issue that both users and media organizations are consistently facing. Second, think about [...]
[...] next phase of filtering will center on the evolution of the “Semantic Web,” which Ashish Soni, who directs the [...]
[...] could use metadata to make information they produce more useful and valuable (further defined in Vol. 2, [...]