Ideal Jobs for CrowdFlower
CrowdFlower combines the best of human and machine intelligence to enrich data for a variety of use cases. You can build interfaces relevant to your specific data by creating jobs. Some key characteristics for ideal jobs for the platform are:
- Cannot be fully automated
- Can be divided into discrete steps governed by objective rules
- Can be carried out online without requiring contributors to leave their computers
Here are three elements that qualify certain jobs for CrowdFlower:
1. Humans with Machines
While models can predict confidently in some areas - humans can help raise confidence in those areas that computers are lacking. Ideal jobs are when computers can automate a portion but not all of the data, thus requiring a human-in-the-loop workflow. In this environment, computers can complete the high confidence rows and humans the lower confidence.
2. Objective Rules and Clear Instructions
Most jobs involve a series of tasks that are accompanied by objective guidelines for contributors to follow. For example, in a search relevance job the contributor is asked to compare a search query to a product title:
In this example, the contributor must understand what makes a query “Excellent” or “Poor”. It is essential to have clear instructions and rules for the contributor to follow in order for them to be successful and return accurate data. The better able you are to teach contributors your guidelines, the more accurate and valuable your results will be.
3. 100% Online
The most successful CrowdFlower jobs have something in common: they all involve tasks that keep contributors at their computers. When contributors are asked to collect information from the real world, collaborate with others, or turn to another device to carry out a task, extraneous factors can skew results and unforeseen obstacles are more likely to interfere with task completion. CrowdFlower only supports jobs which can be completed online. An example of a task not supported would requires contributors to use a smartphone to provide feedback on mobile applications.
Common Use Cases
The ideal jobs for CrowdFlower are mainly objective in nature, but also have subjective nuances that prohibit complete automation. These jobs are summarized as low complexity but high volume and too time consuming to complete on your own. If your job has a repeatable objective set of rules that any contributor can learn, you’re likely have a great fit for CrowdFlower.
For a better understanding of some of the most common use cases, consider the examples listed below:
- Sentiment Analysis – interpret context clues to extract meaning, mood, intention, or tone from a image, tweet, video or a piece of text.
- Categorization – Categorize companies, products, media, or any number of other items, based on the criteria that you provide.
- Content Moderation – Moderate your content for guideline violations, inappropriate content, or spam. Whether it’s text, photos or video, the contributors can evaluate the content against a set of objective rules.
- Business Listing Verification – Improve the quality of your business data by verifying company websites, addresses, phone numbers, and other important fields.
- Data Collection and Enhancement – Collect new data or enhance your current databases by finding social media handles on LinkedIn or Twitter profiles, locating parent and subsidiary companies of a business, or by flagging duplicate business records.
- Search Relevance – Help refine a search algorithm by rating how relevant search results are to a query.
- Image Transcription – transcribe text from images.