Spark Plug #15: Recruiting Participants
Stuff worth sharing for research and learning technologists
Last week something interesting happened. The Verge published a story about this teenager in America that posted a tiktok video sharing her minimum involvement ways to make money online. The post went viral š¦ and tons of young women signed up to Prolific, one of the recommended sites in the video.
Prolific is a participant recruitment platform that focuses on supporting researchers; it started out of Oxford in 2014 and was backed by YC in 2019. If you speak with anyone and I mean anyone that tried Prolific for their own research, you will hear a ton of excitement in their voices. A researcher can expect to take anything from 1 month to more than a year to complete the data collection process. With Prolific, they get hundreds and thousands of respondents in less than a few hours; thatās what this excitement is all about. And itās really easy to set up a survey on Prolific, unlike adaptingĀ to Mechanical Turk.
The biggest challenge with all participant recruitment platforms (Iāve got a list here), be it via Mechanical Turk, Prolific, Facebook ads or manual outreach, is having a really good and robust sample of the population, but not just of any population, more specifically the one being studied. To ensure the sample is robust and in line with research objectives, an academic will most certainly define some screening criteria.
š For example, Mechanical Turkās demographics vary and have been evaluated to be more diverse than US College Campus students (š»this is a good thing!). Many developed a series of screening techniques and exercises for best practice when using MTurk. Similarly, recruiting participants offline has proven to bias at least 60% of studies in psychology and human behavior, as most of these were āfrom Western, Educated, Industrialized, Rich, and Democratic (WEIRD) societiesā. Bottom line, skewed demographics in a research study is not a new thing, itās a challenge that social science researchers have been dealing with for ages.
š¤¦āāļøWhat suprised me about this particular example is that nobody seemed to ride the wave of increased participation from teenage girls (I am sure there are plenty of studies that look at young adults), or maybe nobody is talking about this!
Other researcher challenges:
Famous behavioral scientist and WSJ columnist Dan Ariely was under fire for a case of academic fraudš„·. If you remember that study concluding that you only need to sign a form declaring to be honest in order to decrease cheating behavior, well thatās impossible to replicate!
Before student startup accelerators were a thing, universities had incubators or at least a legal person to support its academics filing for patents and collect royalties. This chronicles the story of the professor at University of Florida who was refused an upfront fee for the rights to Gatorade back in the 60s, and now the company is still paying the university around $20M per year! š¤ Ka-ching! š¤
Recently ruled in the US: artificial intelligence cannot be an author on a patent. A decision I will applaudš , we donāt need to blur any more lines between humans and machines.
Check out this extensive review of ResearchRabbit, a literature search tool, by Aaron Tay š
CourseTech
Amazon will cover its employeesā college fees, in the US.
Class 101 raises $25.8M out of Singapore to focus on arts and crafts education. Similar to Spark Studio from the YC2021 batch, but for all ages and with more advanced courses.
Ahead of its IPO next year, Byju just dropped $200M cash for Tynker, a coding platform for kids; hello acquisition number 10!
Udemy is serious about going public at a valuation between $6M and $8M. Courseraās listed at $33/share for $4.3M earlier this year, and was valued by the public at more than $7M in the first week, but the price keeps falling. Not a good prospect for Udemy.
See you next week!