Follow

Automatic Job Pauses - Why do They Occur?

There are a few reasons why a job may autopause:

  • Out of Funds
  • Alerts Setting

 

Out of Funds

A job will pause automatically if the job has run out of funds. This is a preventative measure implemented in the CrowdFlower platform. If a job runs out of funds, it is most likely one of the following reasons:

  • Poor test questions
  • Dynamic judgments and high disagreement
  • Over-collection due to manual rejections

What do these reasons mean?

All of the reasons listed above increase job costs due to judgment overcollection. More specifically:

  • Poor test questions
    • When test questions are of poor quality or capture edge cases, which are use cases that do not capture the majority of the dataset well, contributors may not answer these test questions correctly, affecting their accuracy and potentially changing their judgments from trusted to untrusted. Untrusted judgments in Work Mode are accounted for in job cost and will increase job cost because of the over-collection process. the job will need to collect more trusted judgments to remedy higher worker disagreement in order to finalize units.


      Over-collection Process

      If a contributor's Work Mode judgments become untrusted, the worker agreement for each of those units the contributor worked on will decrease and the units will need to more trusted judgments in order to finalize.

  • Dynamic judgments and high disagreement
    • Similar to poor test questions, dynamic judgments and high disagreement will increase job costs due to over-collection.
  • Over-collection due to manual rejections
    • Manual rejections will cause contributor's judgments to become untrusted, which will lead to judgment over-collection as well.

For more on job costs, click here. For more on billing, click here.

To resolve this issue, add more funds to your account and manually resume the job.



Alerts Settings

Another reason why your job might pause is because of the default alert settings in a job. Each job has pre-set alert settings, such as high Test Question failure, that will pause your job as a preventative measure. You will receive a notification if your job is paused due to an alert setting.

To resolve this issue, please contact help@crowdflower.com as this is an antiquated setting that the support team will need to change for you.

 

A job may also be paused by our support staff in case of suspicious activity. Our support staff will reach out to you if this occurs.

 


Was this article helpful?
0 out of 0 found this helpful


Have more questions? Submit a request
Powered by Zendesk