Follow

Getting Started on CrowdFlower - An Overview to Building a Job

 

The CrowdFlower platform supports many different use cases, such as sentiment analysis, search relevance, data categorization and content moderation. Before you begin applying your use case to the CrowdFlower platform, we provided some tips and tricks to help you create a successful job that will yield high quality results. If you need more information, please see these articles.

Adding Data

  • It is important to format your data before uploading to the platform.
    • It is best to use a UTF-8 encoded .CSV, however .TSV, .XLSX, and .ODS file formats are also acceptable and require UTF-8 encoding.
    • All data files should:
  • To view more related articles, click here.

Design

  • Graphical Editor
    • Allows you to:
      • Create or modify job title
      • Insert Data
      • Generate questions without typing any code.
    • Best used for those who are not familiar with CML.
    • Can be used to customize templates or validators to ensure inputs are entered correctly, and logic to hide or show certain questions.
    • To view more related articles, click here.
  • Code Editor
    • The Code Editor is used to build custom jobs with HTML, CSS, Javascript, and Crowdflower’s markup language, CML.
    • It can perform all functions as graphical editor
    • To view more related articles, click here.
  • Instructions
    • Effective instructions in your job will help the contributors understand and correctly complete your task.
    • They should cover the steps to complete the job, as well as helpful tips and examples.

Quality

  • Test Questions are:
    • The most important quality control mechanism in the CrowdFlower Platform.
    • Rows that are pre-labeled with known answers that are randomly inserted throughout your job.
    • Will be used to test and track contributor performance.
    • Ensure that only the contributors demonstrating competency in your job are allowed to submit judgments.
    • To view more related articles, click here.

Settings & Launch

  • The Launch page contains settings for:
    • Judgment per row
    • Payment per row
    • How many rows are going to launched for that particular run.
  • Definitions and other common settings can be found here.

Monitor

  • Job mornitoring allows you to
  • Check on the health of your job
  • Check on the health of your job
  • Identify areas for improvement.
  • Reviewing your dashboard provides a high-level overview of the job.
  • By far the most important aspect of monitoring the job is reviewing your test questions.
  • To view more related articles, click here.

Results

  • After your job has completed you can:
  • Edit report settings
  • View your results
  • Audit the output data
    • Note: The audit is extremely important for iterating on your job, as it allows you to find areas of opportunity for improving contributors’ understanding of the task.
  • To view more related articles, click here.

Was this article helpful?
2 out of 3 found this helpful


Have more questions? Submit a request
Powered by Zendesk