Arbitration Game

[rprentki]

We’re rolling out a new feature tentatively called the Arbitration Game (care to help us come up with a less Lawyer-y sounding name?  Please?)  We’re rolling this out to a few people at a time so we can test it and figure out just what to do with it.  

If you’ve gotten a chance to try it, what do you think?  

Do you like it?

How should we score it?

What should we call it?

[mkwak]

I tried it out for five minutes, and it’s kinda uncomfortable.  Since only part of the neuron is shown at a time, the successive pieces are difficult to ascertain whether those belong.  Also, I’m getting afterimages when I look at the colors, so it takes me a few moments to adjust.  And, adding a button with “Not sure” or something like that helps too.

Right now, when I get a cube that I don’t add to, I just abort instead.  Not sure whether others (the select minority) are doing the same.

[rprentki]

Thanks for the feedback.

Do you get after images when you’re playing normal EyeWire, or do you think it’s the yellow pieces causing it?

[smalljude]

So far so good. No afterimages for me thankfully. My second image is a monster, and it’s hung the page on Chrome (Mac OS X) a couple of times. I noticed that the selection moves up and down through the image in random order… rather than proceeding all in one direction. I think I’d prefer if it was just working its way through without the constant jumping (unless there is a specific reason for it?).

I like @mkwak’s idea of a Not Sure button - just for those very difficult occasions. 

As for scoring, that’s a toughy. I haven’t any useful thoughts on that yet.

[smalljude]

Forgot to mention, I do really like it!  

[smalljude]

OK… seems I can’t get beyond a certain point in the big neuron. It always hangs the page in Chrome and I have to kill it.  Many of the things that come up as controversial are right in the middle of the neuron - not sure why it wouldn’t have been marked by previous players (unless it was too heavy for most people to load and mark at all).

One small thing - I turn the shading/colour on and off while checking. Once I’ve done this, the arrow cursor disappears and only shows up when I move the mouse. Small bug… but a little annoying. 

[norm_rhett]

The email link “https://play.eyewire.org/?mode=1” takes me to the regular game. How can I see the arbitration game?

[balkamm]

@smalljude Sorry you are having crashing issues.  What browser/os combo are you using again?

norm rhett: The link you provided takes me to the arbitration game.  Are you sure it doesn’t work for you?  Make sure there are no extraneous characters in the url.

[mkwak]

I can’t seem to get back to the arbitration game.  One thought is that if the entire neuron is shown, blue + yellow, then clicking on a portion would subtract…?  Then press “submit” or "pass"

Not sure how the scoring would work though.  Maybe by how many of these are reviewed?

[rprentki]

@mkwak and norm rhett

The link seems to be working fine now, I was having trouble with it last night as well.

@Smalljude you’re right that a lot of the stuff highlighted is really not controversial at all.  The pieces that you are arbitrating are stuff that people missed.  So on a challenging piece made up of lots of little bits it’s easy to see how someone could miss one (or several).

@mkwak I’m not sure I follow, which pieces do you want to subtract, the blue or the yellow?

One of the things we want to do with scoring is figure out a way to reward people who do this accurately, i,e. not just hit “keep” for everything.

Do you guys think players who are pretty new to EyeWire would benefit/be able to play this?

[mkwak]

What I mean is that the portion I was looking at, the yellow part extended for awhile, so one section at a time was asked whether to keep or discard.  It would be nice to see the entire yellow part to accept all of it.  

Would we be seeing the same pieces as before or would this substitute the original game?

BTW this forum can be seen by all

[norm_rhett]

Using three different browsers (Chrome, IE and Firefox) “https://play.eyewire.org/?mode=1” is immediately translated to “https://play.eyewire.org/overview.php”. Any suggestions what else I might try?

[balkamm]

@smalljude Oh yeah, forgot to mention, it’s not supposed to be moving through slices in random order.  It’s supposed to be jumping toward it with an exponential decay.  It shows a few slices on the way, and more as it gets closer to it’s stopping point.  If it is truly jumping around randomly, that is a bug.

@mkwak We are grouping segments which may potentially be added together, but only if they are all physically connected to each other.  If there are a lot of different little pieces, we wouldn’t be grouping them.  This would be another game which runs in parallel to the original game.

[balkamm]

norm rhett: Are you logged in?  If not, go ahead and log in and try it again.

[smalljude]

I’m using Chrome on Mac OS X (10.7.4).  It worked fine for the first few, but then the giant neuron that took up most of the 3D view crashed it. Interestingly, it went through a huge amount of yellow sections before the crash. I confirmed that it does move randomly up and down… I was watching the 3D view and it was all over the place. 

I can imagine that people new to Eyewire might find it interesting. It may possibly give a good idea what to look for and how the majority of people are marking. 

I wish I had some suggestions for scoring. It’s hard to think how it could be done in a way that is fairly accurate. 

[smalljude]

Just an FYI … I pressed restart and a different task loaded up - taking me back to the regular game.  Are you aiming for restart to work in the arbitration mode?

[system]

Oops that’s a bug, restart is supposed to work in the arbitration game. Thanks smalljude!

Also, ordering the segments from bottom to top (or similar) is a great idea.

Sometimes the segments are ordered in a particular way (for instance, you say one segment belongs, then the system might ask you whether a child of that segment belongs). In general though, there are lots of individual pieces of dust which drag you all over the volume, which we can fix.

I’m curious, how much time are you guys spending deciding on dust pieces rather than pieces that are part of branches?

[norm_rhett]

Ah yes, logging in first. Perhaps the AG should start there.

Arbitration is rather tedious. My first task was “keeping” a lot of fill-ins that I would guess took much longer than the original task. I’d like to be able to submit an exceptional case such as a low score for an obviously correct addition that others have missed. If logged in and agreeable, they, and possibly others, could be invited to review the evidence and just comment or discuss it. A fixed reward of some points that doesn’t encourage collusive gaming of the rules might encourage participation. If this results in productive collaboration, so much the better for learning and science.

[smalljude]

Sometimes the segments are ordered in a particular way (for instance, you say one segment belongs, then the system might ask you whether a child of that segment belongs). In general though, there are lots of individual pieces of dust which drag you all over the volume, which we can fix.

I'm curious, how much time are you guys spending deciding on dust pieces rather than pieces that are part of branches?

Sorry for a stupid question, but what exactly do you mean by dust? 

[rprentki]

By dust we mean those tiny tiny pieces.