December 31, 1970 4:00 pm
Community Blog
December 21, 2009 2:42 pm
Today, I'm pleased to report we're rolling out new work and new features.
Here's a quick update:
New Features:
Credentials: We are adding our long-awaited Credentials feature this week. Some projects will be reserved for workers with the appropriate Credential, which represents a specific level of skill in a given area. You can earn Credentials by taking tests. Often, the test is simply to do a single task of the type the Credential represents. Your work will be reviewed and a decision made whether or not to award the Credential.
The Credential process will help ensure that your hard work is being reviewed by qualified workers. After testing for a Credential you will receive email notification when a decision is made. Generally, you can only test for a Credential when we have active work available that requires that Credential. --Also, if you fail the test for a specific Credential you may be able to take that test again, but you will have to wait.
Reasons: Technically, this isn't brand new, but many of you will be seeing them for the first time in the next month. Reasons allow a worker to specify why they are rejecting something. They decrease the likelihood that a rejection will be unfairly graded.
Highlighting: If you do one of the tasks for our new Highlight Resumes project, you'll see our snazzy new highlighting widget.
Workforce and Work: Today we expect to post 6 projects of work. We still don't have the quantities we want, but the backlog is building nicely. Thank you for your participation!
Respectfully,
Alex Edelstein
CEO CloudCrowd
November 13, 2009 6:41 pm
Dear CloudCrowd Workers,
Last week we successfully implemented our first significant piece of work for an outside client. If you did any of the work involving sporting goods, you know the project I'm talking about. Our client was very pleased with the quality of the work and the speed of our response, and that portends well for our ability to acquire future work. Thank you to everyone who worked on this project for doing a great job on a challenging task.
New Features:
We recently added the ability to view the details of past work you've done. If you haven't seen that yet, go to your History and try clicking on one of your tasks. This feature should make it easier to understand why tasks were rejected.
We also developed a Categorization widget that Workers used during the sporting goods project to categorize different products. The tool received positive reviews, and we'll be using it again in the future.
Our new corporate website at
www.cloudcrowd.com has gone live, and we got our company and community blogs up and running. We made some behind-the-scenes improvements to our payment system as well.
Workforce and Work:
Meanwhile our registered Worker base passed 6,000. We're very pleased to have such a large number of Workers, but I continue to be frustrated at the bottlenecks holding up the delivery of more work. We're making progress, though. We made a couple of great additions to our development team, which will significantly increase the speed with which we can deploy new functionality and projects.
We have a number of new projects already lined up, and we'll be delivering new work soon.
Thanks again for your patience.
Respectfully,
Alex Edelstein
CEO
CloudCrowd
www.cloudcrowd.com
October 28, 2009 2:51 pm
Dear CloudCrowd Workers,
I'd like to give you another update on progress. You have all been very patient as we implement various infrastructure improvements.
Credibility
I'd like to talk first about Credibility. Today, we finished recalculating everyone's Credibility using revised algorithms that we believe are significantly more accurate. In about 33% of the cases, we adjusted a Credibility down, and in about 33% of the Cases, we adjusted a Credibility up. We don't like doing this kind of arbitrary change, because we know it's jarring, and we hope to avoid it in the future.
Why do it at all? Well, we're trying to address a very fundamental issue: Bad work needs to be rejected, and Good work should be approved. That isn't happening enough yet. Many of you have shared with us the frustration of getting work that you believed was absolutely correct rejected. When that happens incorrectly, it's often because someone who disagreed with you has an overly generous Credibility rating.
The main solution is to tighten things up. If you earn a Credibility of 80 and you're in disagreement with someone with a Credibility of 30, you will "win" the decision. If they have an improperly earned Credibility of 90, you're in jeopardy of losing.
As a result, we're continuing to evolve both the basic Credibility engine and the tools we use to improve it. One of the biggest new tools is what we're calling Check Questions
Check Questions
On Tuesday, we started to inject a small number of "canned" questions into the mix. Each Worker is programmed to get a small number of these Check Questions mixed in to their regular questions. Each CQ has been carefully evaluated, mainly to determine if it's sufficiently unambiguous. Most CQ's involve testing to see whether a submission that should be Rejected is properly Rejected, but some involve testing whether, for example, an Excellent piece of work gets a Good or Excellent grade from its Reviewer.
The penalty for failing a Check Question is tough: we reduce your Credibility to 25 and put you on Probation. Our hard argument is that if you're following the directions, you won't have any problem ace'ing these questions. If you get them right, we'll pay you for them just as if they were real questions
Review & Appeal
But what if you think we were wrong? We are getting close to implementing an Appeal system that will allow you to appeal the ones you think were Rejected wrongly. We're also about to put in a basic solution you can use to view your old work, reviewing both the answer you gave and the answer the Crowd concluded was right.
Big Drops in Credibility
One thing that has been noted is how severely Credibility can drop. There are two ways you can lose a great deal of Credibility all at once. One way is if you blow a Check Question. The other is if an Administrator happens to hand-review a piece of work you did, enters an answer, and the answer disagrees with yours. In both of these scenarios, you can lose 75 points instantly. The logic in both is the same. If you're in serious disagreement with known right answers, we can't count on your work as much as we had thought. You might ask "what if the Administrator is wrong?" It doesn't happen much, because the only people doing Administration right now are very strong on this subject matter. However, it has been seen a couple times to happen. That's another good situation where Appeal will be useful.
Upcoming Projects
By the end of this week, we will post a new project which has to do with finding online data to populate a catalog. We also should have a lot of new tasks, as soon as we get some of the above items in place. In addition, we're working on a feature we call Credentialing. Once we have that, we'll be able to get the Rewrite tasks back up.
New Blog & Forums
I'm pleased to report that we have gotten our Blog up. Information like this letter will be posted to the Blog, and we'll be shifting our forums from Facebook to our own forum setup.
Overall
Thanks again for your patience. We really aren't set up to handle this large a Worker base, and it shows. We're delighted to have this level of interest, and we hope you'll be patient as we continue to make major improvements. I am convinced that we will build this into a huge platform offering an incredibly rich selection of tasks and projects to Workers. Now if we can just get through our childhood and adolescence!
Respectfully,
Alex Edelstein
CEO
CloudCrowd
October 26, 2009 5:49 pm
I wanted to give everyone an update on the work we're doing to improve the CloudCrowd work experience. We have been building a bunch of pieces of unglamorous, but necessary, infrastructure. We completed the part of the Payment System that chases down all the little payments that fail or are denied. That's why your "Awaiting Payment" field now shows the right value (that was a pretty embarrassing bug).
We also built the Notification system we needed to provide better information to workers. For example, we have a concept of Probation, which occurs when a Worker's Credibility drops to a certain point, but before they would be Suspended. That requires some automated emailing, which is now in place. We made good headway on getting our Blog up, which will make it easier for Workers to get up to speed on what's going on. We made our Referral code smarter so your referral links and invitations will work consistently.
We redesigned the questions for the Projects some of you have worked on, to provide ways for inappropriate input data to be flagged. Finally, we fleshed out our Credibility model, adding some useful extensions that should help weed out fraudulent work.
We expect to add new work and a new project this week.
Alex
October 16, 2009 5:53 pm
We wanted to give you an update on the many things going on at CloudCrowd. We're delighted at the number of high quality Workers that have signed on to try CloudCrowd-based work. We're frustrated that we don't have a more complete, more full-featured experience for you yet. And we're hard at work improving things.
Some interesting stats:
We're processing more than 1 task every second when we have tasks available. The leading Worker has earned more than $270.
There are also things that we are NOT happy about. Too many good answers have been rejected when they should have been accepted. That's a double blow: The good Worker loses both money and credibility. More on what we're doing about that below. The other thing we don't like is that we're having to ration work right now because the demand is so high. Our goal is to allow a motivated Worker to work around the clock, choosing at any given moment from a rich range of Projects. Right now, though, we only have a budget of a couple of thousand tasks per day, and that's getting consumed very quickly.
We're going to be operating the site at about half-throttle for the next few weeks, while we continue to implement a number of technologies intended to reduce false rejections and improve overall quality. We describe some below.
Reducing Inappropriate Rejections
A number of Workers have expressed concerns that everything they review as “weak” or “poor” is rejected. One issue that has been contributing to this perception is that when you do a task and it’s rejected, you are given a grade of “poor” or “weak”. Those grades were showing up in Worker histories without an explanation that the grade was for your work, and was not the grade you gave the worker who did the original task. If you keep this in mind it should significantly reduce the disconnect you may have had between your grading and your approvals.
There are still some cases where work is being rejected even if it’s done correctly. We understand how frustrating it is to submit a review you know is correct, only to have it instantly rejected. Review work is rejected quickly when the grade you give is significantly different from the grades already given by other workers who reviewed the same work. If the other reviewers for a task give inaccurate grades, but all agree with each other, your correct review could be instantly rejected. We have taken a number of steps to prevent inaccurate work from polluting the system, and you should notice a drop in inappropriate rejections over the next couple of weeks.
Appeal Process
We will soon implement an Appeal feature, which will allow you to request a review of your rejected tasks.
Payments
We changed our payment schedule to make it easier for us to make payments on time, every time. Work approved by 5:00PM Pacific Time will be paid by midnight of the next business day. We appreciate that no one wants to wait longer to get paid, but with the huge growth we've experienced we slowed things down just a little to give us time to review things before payments go out.
Notes on Quality
Keep in mind that we need high quality work from every Worker for every task. If you submit tasks with unacceptable results, you may be subject to suspension with no payment for unapproved work.
Some of the common reasons that Workers have been suspended:
a. Submitting rewrites for the wrong site, or merely pasting in the original text.
b. Failure to compare a rewrite to the original text.
c. Multiple rejections where it is clear that the Worker has made little or no effort to submit
a correct response.
If you're not confident in your ability to provide quality work for a specific task use the "skip" button to pass on that task or use "exit" to try another project.
Thank You
We are a small company, and we're honestly overworked right now, but we're trying really hard to be as responsive as possible to all of your concerns. Thank you again for your support, suggestions, hard work, and patience. We are dedicated to addressing your concerns and
improving your experience at CloudCrowd.
Respectfully,
Alex Edelstein,
CloudCrowd
October 13, 2009 5:53 pm
More than 7,000 of you have installed CloudCrowd in our first three weeks, and we expect that number to grow even more dramatically once we implement our newest features and release additional work. With our dramatic growth, it’s important to have a place where you can learn more about what’s going on with CloudCrowd. This blog will be the first place to look for the latest news on upcoming work, new features, and any changes or enhancements to the system. We’re excited to have built a dedicated and motivated team of Workers as quickly as we have, and we look forward to growing CloudCrowd along with you.
Alex Edelstein
CEO
CloudCrowd
Showing 1 - 6 of 6 Articles
< Previous 1 Next >
Write a comment