Request for plaintext statistics

Feature Requests
#1

I’d like to request that stats be accessible as plaintext (and therefore scrape-able) in the page source for a user’s profile.

It would make it simpler for third parties to compile statistics, lightening the burden for the Eyewire developers.

As an example, the ‘Folding at Home’ project which similarly relies on volunteers to perform scientific research (via donating computing power) exposes users’ stats simply and this has given rise to sites like Extreme Overclocking providing very fine-grained historical data.

This is great for setting up competitions and teams and other simple motivators for participation.

(It also appeals to me to be able to scrape and record my own data!)

Regardless, thanks for setting up a great tool/game!

1 Like
#2

Hey Ruthalas,

This isn’t quite as convenient as what you’re talking about but it’s close.

Try pinging the following URL with a logged in session cookie:

http://eyewire.org/1.0/player/USERNAME/stats

You’ll get some useful profile information in JSON format. :smile:

Cheers,
Will S.
EyeWire Developer

2 Likes
#3

My god…

I could use this to create forum signature banners!

Let the PHP begin!

EDIT: How often does each user’s stats get updated? Or are these values real time?

2 Likes
#4

That is perfect! May thanks for the direction sapphiresun!

1 Like
#5

Okay, I’m having a bit of trouble understanding the fscore array. To my understanding, accuracy is determined with the formula tp/(tp+fn), right? I tried using the numbers on my own stats, then comparing it to what is displayed in the EW interface, and I get slightly different values. Am I supposed to do something with fp as well, or is that just there to assist in calculating over/under colouring?

1 Like
#6

Hey Higaran!

Happy to see you again. :slight_smile: I remember you showed me your sick setup a while back.

You’re partially right. tp / (tp + fn) is a measure known as “Recall”. 1 - recall = undercoloring.

tp / (tp + fp) is another measure called “Precision”. 1 - precision = overcoloring.

Your accuracy is a measure known as fscore (or F1) which is the harmonic mean of precision and recall. If you’ve ever wired circuits in parallel and computed the resistance you’re familiar with this setup although there’s an odd factor of 2 just hanging out in this one:

1 / 2f = 1 / p + 1 / r
f = 2 (pr) / (p + r) EDIT: This used to be flipped incorrectly.

Where f is your fscore, p is precision, and r is recall.

Hope that helps!
Will

1 Like
#7

Glad that helped Ruthalas! The values are realtime Hiigaran.:smile:

1 Like
#8

Hmm, I tried following the formula and plugging my own fscore data into it, but it’s giving me an incorrect value. I must be doing something wrong…

"tp":2033804,"fp":142702,"fn":79114
=94.8%

Precision: tp/(tp+fp)
1 - P
P = 2033804/(2033804+142702)

Recall: tp/(tp+fn)
1 - R
R = 2033804/(2033804+79114)

f = 2 (p + r) / (pr)
2* ( (1-(2033804/(2033804+142702))) + (1-(2033804/(2033804+79114)))) / ((1-(2033804/(2033804+142702))) * (1-(2033804/(2033804+79114))) )
=84%?

Taking the fscore data directly from the JSON, so those are definitely correct, and im taking my accuracy value from the EW interface. Since it is real time, it should be showing 94.8%, but when I try to follow the formula and plug the values in on the second last line, I get roughly 84%.

1 Like
#9

Whoops. Sorry, I made a boo boo in my above algebra.

F = 2 * (P * R) / (P + R)

I forgot to do one of the inversions (I was doing it all in my head while I was waking up haha).

Here’s what I got from the computation:

> tp = 2033804
2033804
> fp = 142702
142702
> fn = 79114
79114
> p = tp / (tp + fp)
0.9344352829718825
> r = tp / (tp + fn)
0.9625569946396405
>f1 =  2 * p*r / (p + r)
0.9482876955041049

It’s also important to note that in your calculation you’re using the values for undercoloring and overcoloring. Those are merely derivative values from precisions and recall that make it easier to understand your performance. The F-score is computed from precisions and recall directly.

Hope that helps!

  • Will
1 Like
#10

For what it’s worth, here’s the JavaScript code that computes your F-Score for the progress bar in EyeWire:

fScore = function (args) {
	args = args || {};

	var tp = args.tp;
	var fp = args.fp;
	var fn = args.fn;

	if (tp + fp + fn === 0) {
		return {
			fscore: 0,
			precision: 0,
			recall: 0
		}
	}

	var precision = tp / (tp + fp);
	var recall = tp / (tp + fn);

	var fscore = 2 * tp / (2 * tp + fp + fn);

	return {
		fscore: fscore,
		precision: precision,
		recall: recall
	};
};

The fscore looks a little weird because, in my great enthusiasm to more fully understand the fscore years ago, I simplified the algebra to reduce it to tp, fp, and fn. Nowadays I probably would have just done it in terms of precisions and recall as it’s easier to remember and verify. Either way is fine.

1 Like
#11

Hmm, I’ve implemented a PHP variant of your JS as follows:

<?php
$tp1=1526081;
$fp1=88046;
$fn1=77904;
$tp2=2033804;
$fp2=142702;
$fn2=79114;

$accuracy=(((2*($tp1/($tp1+$fp1))*($tp1/($tp1+$fn1))/(($tp1/($tp1+$fp1))+($tp1/($tp1+$fn1)))*100))+((2*($tp2/($tp2+$fp2))*($tp2/($tp2+$fn2))/(($tp2/($tp2+$fp2))+($tp2/($tp2+$fn2)))*100)))/2;

echo $accuracy;
?>

I could probably simplify $accuracy a bit, but I have another issue to deal with first. This code works fine, and outputs the right value for 95% (if your eyes haven’t crossed at my lack of finesse with coding, you’ll see that I’m getting the average accuracy of L1 and L2). However, putting the $accuracy line into my main script gives me 98%.

So far I know that the formula works. I also know the data is correct, since I copied the values above from the local file I saved to the server. There must be something in my code, between the point where the data is read, to the point where the data is displayed, which is causing the value to be incorrect.

The search continues…

EDIT: Found the cause. My stupidity. I was missing two of the six variables. IT WORKS!

2 Likes
#12

Haha, congrats! Let me know if I can help you guys with anything else.

Just as a tip, being neat in your coding from the start nearly always pays off. :wink:

1 Like
#13

I have another question for you!

What does the discrepancy between the last line here indicate:

My data has all zeros there, while Hiigaran’s is filled with data.

1 Like
#14

Good question! Notice that there are two confusion matricies shown inside of an array [ {}, {} ]

The first one corresponds to difficulty 1 cells (Mystery/Artifact), the second corresponds to level 2 cells (Starburst/Lightning/Relics).

If you look at your profile, it looks like you haven’t passes the level 2 training on this account. Time to get cracking if you want those zeros to turn into something else. :wink:

Will

2 Likes
#15

Ironically, it was in the neater part of my code that the problem was in.

As for that formula, I just threw it together quickly by substituting the formula provided, with the appropriate variables. I plan to improve that line later on though…

1 Like
#16

It’s also possible to access points and cubes data for previous days using this URL:

http://eyewire.org/1.0/stats/top/user/by/points/per/day

Where the ‘user’ can be changed with ‘country’ to get per-country statistics, ‘points’ can be changed to ‘cubes’ to get completed cubes data (although it won’t tell you how many of those were trailblazes) and ‘day’ can be changed to ‘week’ or ‘month’ to get weekly and monthly data respectively.

The URL takes 2 GET parameters: from and to. Both of them are a date in YYYY-MM-DD format and can be used to get data in a specific timespan. For example, the URL

http://eyewire.org/1.0/stats/top/user/by/points/per/day?from=2016-02-01&to=2016-02-29

will return everyone’s points per day for the entire February.

Keep in mind that if the timespan is too large and the response is too big, the server will cut it short, so you can’t go requesting data from 2013 to 2016 in one request.

2 Likes
#17

That is quite useful!

1 Like
#18

After putting the finishing touches on the code, EyeWire functionality is a go!

This one is dedicated to all the Twinkle supporters of today’s challenge!

http://www.bronyathome.org/sigimage.php?e=YOURUSERNAMEHERE&b=twilight1

To ensure that EW servers aren’t bombarded with requests, user data is stored locally for an hour, and then requested from the server after that time whenever the image is loaded.

EDIT: Let me know if anyone is interested in the entire source code.

3 Likes
#19

I am interested in how you specifically grabbed accuracies from fscore.
Edit: Ahhh…I got it. Thanks.

2 Likes