The power and limits of curation — panning for gold in the digital flood

This is the third and final post in our continuing series on how and why The Hawaii Project recommends great books, and more broadly the key ingredients in a good discovery or recommendation system. (Part I: Ratings & Reviews are Broken; Part II: the Limits of Social Discovery).

Today we’ll focus on Curation — the power that talented curators can bring to discovery, the limits of what Curation can do for you, and solutions starting to develop in the market.

We’re all drowning in content. Everything from long form writing on LinkedIn and Medium to snackable content like our Twitter feed or Instagram photos. Eliot Peper said it well:

Blogs made everyone a journalist. Self-publishing made everyone an author. Youtube made everyone a filmmaker. iTunes made everyone a musician. Publishing houses, record labels, and newsrooms have lost their long-held position as gatekeepers of taste.

How do you make sense out of that flood of content? Separate the gold from the rock?

Traditionally, people might use Google to navigate the content. But that only works well if you know what you’re looking for. Googling “Elon Musk Biography” is likely to get you where you want to go; googling “what should I read next” will not. In the enterprise world, Analytics Dashboards and Big Data are great at answering pre-defined questions but not so great at exploring a previously-unstated (and perhaps unknown) question.

People hunger for authentic, relevant content, not listicles of top 20 crap. They’ve begun turning to Curators to surf the flood of content. If Robert Scoble says a startup is interesting, it probably is. Maria Popova at Brainpickings produces a mini-flood of fascinating articles and literary expositions. Jason Hirschhorn’s MediaRedef is an awesome curated set of commentaries on the media industry. Ryan Holiday consistently recommends life-changing books you probably haven’t heard of. And that’s not to mention the still-high-quality “institutional” curators and content producers— the New Yorkers, Guardians and Wall Street Journals of the world. While the economics of media is deeply challenged due to the commoditization of online content, traditional media still churns out high quality content in a number of places.

Even Tech companies are getting into the game. Apple Music leans hard on curators to select fresh, interesting music. And Product Hunt, previously focused on crowdsourcing the discovery of cool startups, is getting into the curation business through their exciting new Product Hunt Books initiative. Many companies are embracing smart content curation as a marketing & branding tactic.

Curation as a discovery and recommendation technique has the following key benefits:

• Curators have great “taste” in a well-defined area. I want to know the startups Robert Scoble is talking about, and if I’m interested in startups I read/listen/follow him.
• Curators see connections and patterns that machines miss. A machine may not see that The Snow Leopard and Zen & the Art of Motorcycle Maintenance are both Zen travelogues, but a perceptive curator will.
• Curation provides “smart” aggregation. Machines can aggregate a lot of content, but may not put the right things together. A human can produce something like — “here’s what Marc Andreessen and Peter Thiel both think are great startup books”.
• Curators have aligned incentives. Retailers often have an incentive to push certain items on you. They may be getting paid by the creator of the item to promote it or have better margins on it. Many media outlets are chasing the controversy, click-baiting you with attention grabbing items. Curators often don’t do it for the money at all — their incentive is to bring you interesting things that align with their personal brand.

OK so if Curation is so great, why isn’t everyone getting their content this way?

Curation has three shortcomings.

• The curator isn’t always for you. (I love Scoble’s taste in startups, but don’t care about the cool new cameras he’s obsessing over.)
• There’s too many of them. (even keeping up with the curators I mentioned above, could take an hour a day, and there are hundreds of thoughtful, interesting people writing — who has time to wade through all that?)
• It requires pro-activity on my part. (Even with Twitter and Facebook and Medium and …, I still have to go visit each of those curator sites to see what they’re writing about, or even if they are writing. I don’t have time for that, and even if I do, I’ll likely forget).

What if…..

What if there was something that knew the curators you valued and could constantly be watching what they’re up to? And telling you when they’ve curated something you’ll be interested in? (RSS readers used to perform the watching function, before RSS died as a consumer tool, but RSS never really helped with the personalization and filtering side of things). Someone / something that would “curate the curators”?

I call a system like that “Personalized Curation”. We’re beginning to see the beginnings of it with Music (Apple Music & Shuffler.FM), News (Nuzzel & Quibb), and that’s what we’re tackling for Books with The Hawaii Project. Domain-specific personalized curation also allows for easy consumption of the discovered content — e.g. when Apple Music brings me an interesting song, I can listen to it.

In a non-domain-specific way, Google Now and Flipboard are also headed that way. But because they are domain-agnostic, they can’t offer quite as deep an understanding of my personal interests and it’s harder for them to facilitate consumption (e.g. I can’t easily listen to music on Flipboard, or at least don’t think to).

Chime in in the comments! How do you deal with the flood information? Are there systems or techniques that work for you? Have you seem someone tackling this successfully in a domain-agnostic way?