Reap and Complete Stats and Scoring Update


#41

I suppose my question about all is this is: Is this really necessary? I’m kind of confused why this rule was implemented in the first place especially since it doesn’t appear there weren’t any communications regarding this rule change up until now, and from at least our perspective it seems like this is a solution without a problem.

If the admins have found that this rule is necessary due to some circumstances that we haven’t experienced then shouldn’t this have been discussed or brought up before any changes were to be made?

It’s clear that this is touched a nerve with many of us so I’d like to at least hear from the admins as to why this is a necessary change in the first place.


#42

I don’t have anything to add to the above, as I feel that the above contributors have done an excellent job communicating the concerns and issues associated with this change. I would, however, like to reiterate my suggestion for a meeting/hangout, to discuss the above.

Cheers!


#43

Sorry guys, this was an accidental deployment! We’re going through responses now.


#44

Problem with a hangout, is how do you find a time where everyone with concerns can attend? I love the idea, but also know with my work schedule it would be very unlikely i could join :frowning:

@amy thanks for checking in! :slight_smile:


#45

me too would not vote for hangouts, i like more forum discussions. Just need time to check all issues arising with new stuff


#46

Well, if it was an accidental deployment, then the point is no longer pertinent anyhow.


#47

Hey everyone,

Thank you so much for your patience while we went through your posts here. After having conferring here at HQ, here is our fuller response.

First of all, we are so sorry about the accidental deployment! We would like to be clear that although we did intend to eventually have Scythe reaps automatically remove SC votes the way that admin reaps do, we did not plan to include this in a release just before the weekend. We had planned to discuss this change with Scythes in advance so that we could immediately address concerns like what you have all raised.

The reason we needed to release this in the long term was that we can only give folks an accurate picture of their SCing by making sure a player’s SC tally only includes votes that are correct. “Scythe Complete” means and has always meant for a Scythe to say, “This cube’s consensus is accurate and does not need changes.” Some longstanding bugs and now-resolved issues (like a lack of Scythe Freeze) have led to Scythe Complete being used for secondary purposes or inconsistent branch review techniques; we have hoped to finally adjust our mechanics to make SC votes only persist for their intended use case. But again, we sincerely apologize that we were unable to discuss and test this before the release rather than afterward. We know that on the surface this is a small change, but deeper down it could alter how some of you may have approached SCing for years.

In an effort to reach better understanding all around, we have identified what seem to be all of your specific concerns with this change, and below are point for point answers from all of us at HQ. Once we have an understanding, the GMs will make sure the Scouts & Scythes Manual is up to date in its phrasing. (And apologies that this is long! But we wanted to make sure we didn’t skip over much.)

Concern #1: In Forts, can people still SC cubes that don’t need to be reapgrown? What happens if someone wants to reapgrow every cube on a branch?

Yes, in Forts you can still SC a cube without reapgrowing it, as long as the AI really did trace correctly. As always, though, please first reapgrow your branch without SCing. Then come back and check it or have someone else check it.

If you are checking someone else’s branch and it’s mostly reapgrown but not SC’d, you should reap problems during the course of SCing it yourself. If you’re checking a branch that’s both reapgrown and SC’d, please don’t reap their SC’d cubes unless they have problems.

Concern #2: What about dust reaping? Couldn’t someone do that maliciously?

If you discover missing dust or merger dust in the course of inspecting a cube, it is currently okay to reap accordingly. Fundamentally the consensus was not correct, and you are within your rights to correct it. At the same time, please do not spend time just reaping dust along the majority or entirety of a branch; and if you believe someone might be doing this, please let us know. If necessary, we may build a way to not auto-remove SC votes for Scythe reaps where the changed segments are under a certain voxel limit, or something similar.

Concern #3: Could/should Scythes get a “nuke” button alongside or instead of the new change?

The “uncomplete” button was always supposed to permanently remove the name(s) of people who had incorrectly SC’d before. It’s unfortunate that this wasn’t working, and thus that this bug may have lasted long enough to seem like a feature. But Scythe Complete was always supposed to work such that the only people who wind up having their SCs “counted” are the ones who correctly state that the cube is complete. Attaching automatic “uncomplete” to all reaping now bypasses that bug; and regardless of when we fix that bug, we suspect a separate “nuke” button outside of admins would prove redundant.

Concern #4: How does this affect Scouts’ Log procedure?

In the future, we will be revising whether/when 2 votes are needed for locking a cube in the first place. Ideally we’d like to make it so that 1 vote locks, and 2 votes are simply required for admins to know the branch is really done.

Until that change can be figured out: we may have requested this before, but going forward, Last Scythe Wins means it is not necessary to log cubes as “Scythe Complete” unless:
a) you want to lock the cube without reaping it,
or b) you reaped it after it already had at least 1 vote and now you need a second vote again.

Most reasons to intentionally lock the cube with 2 votes should be resolved simply by reaping away the problem. If you have reaped a cube and needs no further attention, please just create a log entry to explain what you did but set the status to “Good.”

Concern #5: Won’t duplicate effort wind up being spent?

We aren’t too worried about this. Here’s why: the general procedure for branch review is that one Scythe checks a branch from the parents to the children. They SC and reap as they go. A second Scythe then checks the branch, SCing as they go and reaping any additional issues they find. If they follow the logging procedure described above, then sometimes a third Scythe will need to come in and add their vote for a few cubes, but that’s it. Basically: reap first, then Scythe Complete. When a Scythe adds their SC vote, they’re saying the cube is perfectly traced; therefore it should not need to be reaped afterward unless there is an error.

Concern #6: What about test extensions?

We would like to discourage Scythes from SCing test extensions until they are certain the new branch really belongs. Please log the starter cube as “Watch / Test Extension” and give it some time to grow before SCing. At the moment, if a test extension is SC’d and it turns out that this was incorrect because the extension was bad, these votes are already subtracted from players’ SC totals once the extensions are reaped out and stashed.

Concern #7: SCing takes work, so how is it fair to remove acknowledgment/points for that?

Making sure that we have accurate Scythe Completing is critical to the scientific process. It is the final verification of the cell, no different than making sure we have accurate consensus in each cube. Our points system for normal gameplay also “punishes” players for lower accuracy by giving them less points. This is a core part of Eyewire.

It’s true that by completely erasing old SCs that were wrong, we are not giving people recognition for pure effort. We do give that sort of recognition during normal gameplay with the time/volume component of cube scoring. One compromise could be that all of your SCs that remained at the end count toward a certain number of bonus points, while all of your SCs that were incorrect count toward a lesser number of bonus points.

Concern #8: Some of us prefer to SC and then reap; now we have to reverse it.

Fundamentally it would not be correct to SC the cube before its problems are fixed. However, we know that many of you have wanted a way to SC inside the cube, so we may create an interface change where you can either just reap the cube or reap & SC together.


Hopefully that covers everything that’s been asked about. We would like to conclude, though, by noting that while there are concerns about Scythes improperly reaping to rob others of bonuses, the SC system so far has not been completely fair either. There have been lots of opportunities to SC for stats boosts alone, and not many ways to disincentivize that behavior. Although we deeply regret the mistaken deployment of this change to SCing, we also look forward to dialogue with all of you about how to make SC mechanics better in general.

Thanks for reading, and thanks for all of your contributions to Eyewire.


#48

I have seen/read your response. I’m going to ruminate on it for a day or so, so that I can reply from a place of logic and not emotion. :slight_smile:


#49

The update makes a bit more sense now seeing as it has to do with a more technical feature. Thanks for rolling it back.

I think I lot of our worries can be mitigated if there was a way to be notified of when our SCs have been nuked in a certain cube. This could show up in the notifications and could have an extra option in the settings to enable/disable. Something along the lines of “Your SC for cube #yadayada has been removed” or something.

For me at least the primary concern is that I will lose SCs without knowing, and I will most likely never know since I have no reason to recheck the cube, so if I was at least notified of when this occurs I could investigate accordingly. This could also double as a check on other scythes that may be removing SCs maliciously in order to steal them.


#50

Great idea. Our next project after wrapping these auto notifs will be to officially integrate the SL into Eyewire and that will include a. Lots of discussion with Scouts and Scythes and b. Integration of some parts into the news feed. It won’t be just notifs— toolbar will remain.

In the nearer future, the next version of the cell complete notification will have a tab that allows you to see and jump to every cube that was changed after you Completed it. It will also show this for Reaped Cubes.


#51

Ok, this is my answer to the current dispute.
I have huge problems with the new function, couldn’t sleep last night, and I am still in a churning constitution. Perhaps too early to communicate, but I must get rid of my thoughts. Eyewire got big part of my life as you know, and now there is a big question how I can go on and still keep fun.

The reason we needed to release this in the long term was that we can only give folks an accurate picture of their SCing by making sure a player’s SC tally only includes votes that are correct. “Scythe Complete” means and has always meant for a Scythe to say, “This cube’s consensus is accurate and does not need changes.”

Comment: We only complete when we think a cube is correct. In the new system you punish a scythe who perhaps made a small error by just missing a dust-piece. As you say how to handle dust below, you say dust is not important, so why punish.

Some longstanding bugs and now-resolved issues (like a lack of Scythe Freeze) have led to Scythe Complete being used for secondary purposes or inconsistent branch review techniques; we have hoped to finally adjust our mechanics to make SC votes only persist for their intended use case.

What’s so wrong on the secondary purposes? We obviously loose these now?

But again, we sincerely apologize that we were unable to discuss and test this before the release rather than afterward. We know that on the surface this is a small change, but deeper down it could alter how some of you may have approached SCing for years.

Yes, we did well for years, and now not good anymore.

In an effort to reach better understanding all around, we hav
e identified what seem to be all of your specific concerns with this change, and below are point for point answers from all of us at HQ. Once we have an understanding, the GMs will make sure the Scouts & Scythes Manual is up to date in its phrasing. (And apologies that this is long! But we wanted to make sure we didn’t skip over much.)

Concern #1: In Forts, can people still SC cubes that don’t need to be reapgrown? What happens if someone wants to reapgrow every cube on a branch?

“Wants to reapgrow every cube”? I reapgrow every cube, thought this was necessary and I would do also in the future. Why not reap a cube you are in which is good, you needed time to look into, and get rewarded with 30 points. Reaping and getting the points doesn’t harm anybody. It also shows the scythe-completing scythes after you, that cube has been looked at.

Yes, in Forts you can still SC a cube without reapgrowing it, as long as the AI really did trace correctly. As always, though, please first reapgrow your branch without SCing. Then come back and check it or have someone else check it.
If you are checking someone else’s branch and it’s mostly reapgrown but not SC’d, you should reap problems during the course of SCing it yourself. If you’re checking a branch that’s both reapgrown and SC’d, please don’t reap their SC’d cubes unless they have problems.

?The latter sentence: Nobody reaps a cube with no problems, what you want to say?

Concern #2: What about dust reaping? Couldn’t someone do that maliciously?
If you discover missing dust or merger dust in the course of inspecting a cube, it is currently okay to reap accordingly. Fundamentally the consensus was not correct, and you are within your rights to correct it. At the same time, please do not spend time just reaping dust along the majority or entirety of a branch

; and if you believe someone might be doing this, please let us know.

You say that my way to sc (as explained above) is bad or what? Feel accused

If necessary, we may build a way to not auto-remove SC votes for Scythe reaps where the changed segments are under a certain voxel limit, or something similar.
Might be a solution to my problem

Concern #3: Could/should Scythes get a “nuke” button alongside or instead of the new change?
The “uncomplete” button was always supposed to permanently remove the name(s) of people who had incorrectly SC’d before. It’s unfortunate that this wasn’t working, and thus that this bug may have lasted long enough to seem like a feature. But Scythe Complete was always supposed to work such that the only people who wind up having their SCs “counted” are the ones who correctly state that the cube is complete. Attaching automatic “uncomplete” to all reaping now bypasses that bug; and regardless of when we fix that bug, we suspect a separate “nuke” button outside of admins would prove redundant.

Never missed a nuke button, I am principally against nuking things somebody has done. One loses information and nuking creates uncertainties for us scythes about our past doings
Concern #4: How does this affect Scouts’ Log procedure?

In the future, we will be revising whether/when 2 votes are needed for locking a cube in the first place. Ideally we’d like to make it so that 1 vote locks, and 2 votes are simply required for admins to know the branch is really done.

Sounds ok, but do as soon as possible!

Until that change can be figured out: we may have requested this before, but going forward, Last Scythe Wins means it is not necessary to log cubes as “Scythe Complete” unless:
a) you want to lock the cube without reaping it,
or b) you reaped it after it already had at least 1 vote and now you need a second vote again.
Most reasons to intentionally lock the cube with 2 votes should be resolved simply by reaping away the problem. If you have reaped a cube and needs no further attention, please just create a log entry to explain what you did but set the status to “Good.”

Ok, disadvantage is just: By completing a cell, till now, one could assume that the blue reaped cubes were also double-completed and needed no further look at, so in the future one would have to inspect also these cubes.

Concern #5: Won’t duplicate effort wind up being spent?
We aren’t too worried about this. Here’s why: the general procedure for branch review is that one Scythe checks a branch from the parents to the children. They SC and reap as they go. A second Scythe then checks the branch, SCing as they go and reaping any additional issues they find. If they follow the logging procedure described above, then sometimes a third Scythe will need to come in and add their vote for a few cubes, but that’s it. Basically: reap first, then Scythe Complete. When a Scythe adds their SC vote, they’re saying the cube is perfectly traced; therefore it should not need to be reaped afterward unless there is an error.

Perhaps do not understand all what you say, but in my experience it is quite a lot of double work which could be spent better otherwise (and that’s also a cause I have big problems with the new application – I do not like to do things for nothing)
Concern #6: What about test extensions?

We would like to discourage Scythes from SCing test extensions until they are certain the new branch really belongs. Please log the starter cube as “Watch / Test Extension” and give it some time to grow before SCing. At the moment, if a test extension is SC’d and it turns out that this was incorrect because the extension was bad, these votes are already subtracted from players’ SC totals once the extensions are reaped out and stashed.
Ok

Concern #7: SCing takes work, so how is it fair to remove acknowledgment/points for that?
Making sure that we have accurate Scythe Completing is critical to the scientific process. It is the final verification of the cell, no different than making sure we have accurate consensus in each cube. Our points system for normal gameplay also “punishes” players for lower accuracy by giving them less points. This is a core part of Eyewire.
It’s true that by completely erasing old SCs that were wrong, we are not giving people recognition for pure effort. We do give that sort of recognition during normal gameplay with the time/volume component of cube scoring.

You cannot mix playing and scything. Me personally do more points by playing, and still I do scything because it’s something different and interesting and helping finishing the cells. And honoration for it of course is also important

One compromise could be that all of your SCs that remained at the end count toward a certain number of bonus points, while all of your SCs that were incorrect count toward a lesser number of bonus points.

Why not, ok, problem is still that at the moment one does not know what is the real status, perhaps @Galarun ‘s idea about info over notifications could help.

Concern #8: Some of us prefer to SC and then reap; now we have to reverse it.
Fundamentally it would not be correct to SC the cube before its problems are fixed. However, we know that many of you have wanted a way to SC inside the cube, so we may create an interface change where you can either just reap the cube or reap & SC together.

Would be good to integrate!


Hopefully that covers everything that’s been asked about. We would like to conclude, though, by noting that while there are concerns about Scythes improperly reaping to rob others of bonuses, the SC system so far has not been completely fair either. There have been lots of opportunities to SC for stats boosts alone, and not many ways to disincentivize that behavior. Although we deeply regret the mistaken deployment of this change to SCing, we also look forward to dialogue with all of you about how to make SC mechanics better in general.

I am very unhappy about your saying … improperly … rob…. Never felt a fellow-scythe worked against me. I’m sure, we all do our best, some do more generous, others more accurate, but together we came to the correct result in a rather good time. We all do for science, ok, also for points, but not against eachother!


#52

Thanks for reading and replying, susi. I’m sorry if our response so far hasn’t resolved your concerns and that you’ve lost sleep. :frowning: I can try to address your particular fears more now. It sounds like you are essentially asking:

  • Why can’t Scythe Complete be used for purposes besides saying the cube is really complete?
  • Is it bad now to reapgrow every cube on a Fort?
  • Is it bad now to add dust?
  • How should we know whether to examine a reaped cube?
  • What is this about players robbing other players?

If this is what you’re asking, here are some clarifications that I hope can put your mind at ease:

  • Scythe Complete really needs to not be used for locking cubes that still have problems because when we calculate the final SC counts on a cell, we need to be sure that we’re not including incorrect SC votes. Otherwise we cannot reliably calculate SC accuracy. This is also why we need to be able to automatically remove SC votes that are made while there is still a problem with the cube in question, without re-adding those votes. Otherwise we would be telling Scythes that they voted correctly on things where they actually voted incorrectly. We definitely like galarun’s suggestion for how to keep an eye on votes that have been undone, and we have already planned to give people a way of seeing that information, so that you can learn. And there is still also our suggested compromise of still giving reduced bonuses for incorrect SC votes to recognize the effort. What do you think?

  • No, it is not bad to reapgrow every cube on your own branch on the Fort. I apologize if our earlier phrasing wasn’t clear. Your own behavior on Forts is perfectly fine, as far as I know. The GMs have, however, received complaints about other players spending a lot of time reaping very, very tiny dust into most/all of the cubes on someone else’s branch in a manner that appears targeted. While we do not have clear documentation yet of that behavior, we wanted to assure the players who have complained that it would indeed be strange/wrong for those other players to do. Again, you’re fine! :smile:

  • Going along with the above: generally speaking, it is good to add dust! If you see missing dust in the course of looking at any cube, please do feel free to add it. The consensus is more correct with the dust than without it. We are only trying to make sure that no players dedicate themselves to targeting particular fellow Scythes and adding/removing lots of dust from their traces/reaps out of some personal disagreement (which, again, we have yet to clearly see, but some players appear to be worried about).

  • Knowing when to examine a reaped cube: I would basically say that if it’s on a branch that hasn’t already been double-SC’d, you would want to check it out in the course of SCing the branch yourself. If you are not intending to spend time second-SCing that branch, then obviously the reaped cubes on it can be left for whoever does do the second-SCing. If you see a branch that’s already been doubled-SC’d, ideally the Scythes who already worked on it will have logged the cubes that had their votes undone and need to be re-voted; then the other reaped cubes on that sort of branch can be assumed to otherwise be double-SC’d because the whole branch is otherwise checked. Is this clearer?

  • In terms of players robbing other players, I am truly sorry if our phrasing made it sound like there has been a lot of cheating and unfair gameplay going on. That is definitely not the case. The vast majority of Scythe gameplay in our experience has been quite collaborative and HQ is not that worried about it. We have just needed to respond to concerns fielded from a small handful of Scythes.

I really hope that these answers help! You are a valued player, susi; you trace and reap very, very accurately, you’re very helpful and collaborative, you’re so enthusiastic, and you are not in any trouble!


#53

thanks dj for your fast answer. It calms me down a little bit :slight_smile:

Still, when i reap dust in a cube, and fellow-scythe looses his sc-vote makes me problems (this is actually biggest problem). And i would appreciate very much a solution you suggested to take cubes with low differences to consensus out of the nuke-system.
Informing notifs also good for larger errors, giving me info about bad errors, and giving info about status of amount of cubes i sc-d (for the bonus of course…).

And i think quite clear instructions about this “dust-problem” would be helpful.

Just hope you can add the above suggested adaptations soon.

thanks again, i really love eyewire. for science and fun and personal fullfillment, hope all comes good


#54

Thank you for reading! I’m glad that there are some likely solutions out there to your concerns. We will be working hard to get everything figured out soon. That is another downside to the accidental release of the one change; if it had been accompanied by the other necessary changes, maybe it would have felt less alarming. Once everything is sorted out, we may also do an official video on Scythe Complete procedure to both include in the manual and use as a refresher for folks who may have learned a different way in the past.


#55

I´m very new to EW although I contineously played for a month or two. I really love this game and the idea to combine gaming with science. Yet not have this great experience like most of you have but want to give a “review” of my time with EW.

In the beginning tracing cubes was fun and accuracy-high was the goal. With lvl 2 cubes got more experience and after some time was asked to become scout. But then some problems occurred. I always want to do my best while tracing (so as you). Meanwhile I know where to leave out a merger, or I try to make a test extension. But then often with the result of being the only one who traced this cube that way. So I´m forced to flag this cube, also want to get the points and my accuracy back on the level before. I agree with Atani that it takes some time to get the experience for doing scouting and that maybe only high accuracy does not make someone a good scout. (I think I´m making a good job and thanks to susi and Nik for their
helping hand). But I also noticed that some of the other scouts only play; if I´m wrong, I´m sorry. Only my observation by looking through the entries. As susi said I got more points when I play than scouting a cube. A bit frustrating.

Flagging dust: I´m always in conflict with! What me mostly irritates is, that in normal play I had to be as accurate as possible. If not and leave out a segment (which belongs) I get punished getting less points and accu. On the other hand (as a scout) I was told not to log such tiny dust/nubs. (Also didn´t really know the difference, now I do, thanks to susi). So how to handle this ?
I also understand that it does not make such great difference in cubes´shape if it is missed. Only makes more work for scythes perhaps and gives opportunity in making some points in sc. But for me I mostly cannot ignore it :slight_smile: The cube isn´t finished in my eyes and the earlier I see a missing nub/branch/merger the better it is - for everybody and esp. for the game/science. So I do since I´m scout and feel ok with. But I find it hard to ignore the little nubs missing :wink:

Scouting: As Nik mentioned above since promoted being scout the gameplay changed (for me). Susi adviced me to make manual notes of all cubes I flag. Therefore I start playing by first have a look into scouts log and check this. But there were not all of my flagged ones listed, so I had to type them manually in the inspecting panel. Often then play some cubes, and when I find something wrong flag it. I normally inspect the adjacent branches up and down either. So half the time I spend in scouting, and yes, I like it! Gives me great satisfaction to make the cell complete in my way of playing.

But of course I cannot say much concerning the other things like points depending on sc and scythe-work in background, and the whole way what happened if someone is reaping a cube. Hope you all find a good consensus. We all are different in our way of playing EW but we all do it for science :slight_smile:


#56

It is good that you like scouting @Gruenewitwe and to me it looks like you do a good job and i can understand your consern about not flagging dust and smal nubs. I think we are mainly disencouraging scouts from flagging things like that because as a scout it is more immportant to learn about the big lines and help to mark things that can grow to be potensial problems than do the really fine cheek of branches. and like you are saying missing some dust do not really change the overall shape much and if too many cubes are reaped in a cell it will be more difficault to see where others have been scythecompleting. Also if scouts should flag small things like that it is more work for the scythes to jump to it from log than it is really worth. But if i as a scythe are completing a branch i will often add small nubs that i normaly would wish a scout not to flag. And if i find anything including dust in a cube that is allready reaped i will reap it in because then it has not been any negative effects by doing so, and i would have got 150 pts for it. this will no longer be possiple with new policy.


#57

You are an excellent scout! @annkri described well the situation. In any case you will also get an excellent scythe. Just hope you want to take on the new challenge and that the present problems/uncertainties will be solved soon.


#58

Just to clarify my above statements as to nukes:

I still think, this is an unnecessary implement, you can calculate accu without that and it will create frictions between scythes.

Scythe-complete mainly gave you info about how far a cube has been looked at and gave you information that cubes are ready for final completion. No need for you anymore? You say now, sc must be 100% true which imo can only be said after admin-inspection, thus every sc before not worth much. Could solution be introducing new function “scythe-voted” instead of “scythe-complete”? (lol comes to same at the end, i do just not understand what’s your point)

As long as you do not integrate some changes like taking low-conf-corrections out and give better infos about which cubes you nuked, I tend to not scythe-complete and to avoid inspecting cubes a fellow-scythe has already looked at.


#59

Some potential fixes that might solve the residual problems:

  • Complete from Inspect (would also Reap)
  • Base bonuses on surpassing a certain accuracy - solves dust issue

Scythe Complete is meant to be final. Perhaps part of the problem here is that the feature was being used in a different way than it was intended.


#60

In my option the question is:
Do the admins when completing want to save five minutes because they no longer have the need to cheek cell list for any open tasks in the log and trust completely on the sc vote and lose several scythes? Or do you want the experienced scythes to continue completing and reaping cells and have to cheek the cell list for any open tasks before completing?