No worries! Currently, were sampling about 5 people per task, though sometimes more. We don’t really have a sophisticated process of review, but we’re working on it.
In terms of differences between the humans and the computer, it’s not exactly as
straightforward as all that. Basically what the computer generates is a great big map of the whole volume with probabilities that each pixel is connected to the pixel next to it. We do what’s called
watersheding in order to create the chunks that you guys are working with. What that leaves us with is a probability that each chunk is connected to each of the adjacent chunks.
Now if we want, we can threshold that graph of connections between chunks at a certain probability. That would leave us with the “computer’s choices” as compared with a given human’s choices. The big problem is choosing that threshold. If we choose something too low, then the computer will make lots of merging mistakes until eventually it merges the whole volume into one big blob. If we set the threshold too high, the computer will make splitting mistakes, where it doesn’t join things that should be (the way it looks right now). In the middle, you get mistakes of both types.
For Eyewire, we decide to set the threshold pretty conservatively. The point being that it’s easier for people to fix splitting mistakes than merging mistakes. You may run into a few mergers in the tasks, but they should be few and far between. If you do, abort the task and say that you found a merger.