Comments on tasks for review

I would like to see a comment section for tasks. If im working on a task, but can not finish it, or need a second opinion, I feel as if a comment section is needed, like "im finished, But needs a second look.

I’d second this - there is something similar on the planethunters.org project (Zooniverse group). In this context the inability to revisit a task would present somewhat of a problem, but it is something that could be used in the context of team collaboration.


One issue with leaving comments is something that awood noted in another thread, which is that you don’t want to nudge people into a common solution. Perhaps a way to do it would be to open tasks to a second phase of refinement where you can then see comments and other peoples solutions?


Hey wingnut,


So one thing that you can be sure of is that each task will get a second look.  In general, each task is done by at least 5 people.  From what we are seeing so far, people seem to do a really good job on average.  So even if you aren’t totally sure, just do the best you can.  Other Eyewirerers have got your back!  If you find a task which has some sort of problem or defect or something, you can alert us to that fact by aborting the task.  When you do that, we give you a comment field so that you you can let us know what’s wrong.

@balkamm I see two problems with that.


1. Although that’s probably how it should best work in theory, that isn’t advocated enough. The concept of ‘gamification’ promotes the exact opposite. It leads people to not want to press the ‘abort task’ button. It’s ‘game over’, if you will. :slight_smile:

2. Multiple people can make mistakes. The concept of having multiple people check a certain resource and assume when they agree on a general outcome it is the correct one, only works if the material and environment provided gives the opportunity of achieving a correct outcome. If it’s easy to make a mistake, generally many people will make the mistake, and the outcome could be erroneous. Probably variability in the outcome could be used as a measure of uncertainty, and be a nice indication of areas which need to be thoroughly checked.

Instead, wouldn’t it be nice if the system recognized the uncertainty of people in a less ‘aborting’ fashion, and allows a person to continue finishing a segment, but exit with ‘I’m finished but uncertain’ instead? Ideally he could even indicate the slice he is uncertain about. I’m thinking of all those pesky little staining gaps. Or perhaps you have already established a system which can filter out false positives? Personally I’d rather have a system which at least attempts to point out uncertainties, rather than hiding them.

Nonetheless, it’s a great system, and I hope it scales up well so we can see many new participants in the near future! Meanwhile I’ll stick around and see the current cells form. That new overview is quite astonishing!

Seconded for the need to make a comment on the task. I often don’t want to abort it if I’ve marked a substantial amount but have a question about an image. I’d really appreciate being able to make a comment.

BTW - I pressed upvote on whathecode’s comment above and it went from 1 to 0 … sorry!

While I will re-iterate my support for a comments, don’t underestimate the guiding effect that it can have. A phenomenon that you will see on FoldIT is score grouping by team that shows up in the ‘solo’ score. It occurs where a team member discovers a good position for a ligand that isn’t obvious. All they have to do is show the approximate location in a team shared solution, and suddenly everyone in the team copies it. You will see large numbers of people all of a given team with similar scores. One of the current Flu puzzles is showing this (or at least it will be when they ‘unexpire’ their SSL certificates :slight_smile: )


In a sense, allowing leakage of information from one player to another, at least in a discovery phase of tracing, invalidates the use of crowdsourcing. The statistical independence is a key resource for the validity of results.

Perhaps the best thing to do is try it, and see what happens.

Okay, thanks for the feedback everyone.  One of the things that we are currently working on is a mechanism for revisiting ambiguous parts of cells.  Basically showing you what other people did on a task and asking you if what they did was right or not.  We are hoping that the verification task is easier to do correctly than the search task.  We also think that it should be fairly straightforward to detect the parts of cells that are ambiguous.  As you point out, if no one finds a branch then we can’t really do anything about it but have more people look over it.  Also to address the issue of statistical independence, it would be a separate task done after the fact, so there shouldn’t be any issues with out of band communications affecting results.


In terms of the abort button, I totally get the desire not to give up on a task.  I also appreciate the desire to let us know when you are unsure about what you did.  I think maybe in the mean time we can come up with a semantically less distasteful button name. :slight_smile: