As an academic researcher and a blockchain entrepreneur, I often ponder about where my two passions coincide. In this context, I would like to quickly discuss two aspects related to blockchain and academic research that I think are worthy of discussion.
The first aspect is the process of peer-review: typically, academic researchers disseminate their findings by publishing in peer-reviewed journals. Whether papers are accepted for publication is determined by specialized colleagues who evaluate which research meets the standards of quality necessary for publication. This process keeps changing over time, e.g. due to the emergence of online depositories and working paper series that allow for non-peer reviewed disseminations prior to publication. Nonetheless, the academic career path is still governed by the “publish or parish” maxim, where only peer-reviewed journals count as “publishing” for the purpose of tenure.
A few initiatives have tried using blockchain-based systems to establish alternative peer-reviewed systems such as the project here and you can read an overview of this (very interesting) issue here and an academic paper by Avital (2018) here
Peer-review seems to be a natural blockchain use-case, as (1) there exists the need to somehow store opinions written by various people around the globe and (2) there is potential that someone with an agenda will try to manipulate the data (for my opinion on when potential mistrust justifies using blockchain, see my previous post here). For instance, malicious market players who dislike the implications of a research project for their revenue stream might have an incentive to hack into the system and replace the reports.
Moreover, blockchain can be used not only to store the reports but also to support an effective distribution of tokens as financial. In particular, the concept of Smart Media Tokens (which we upgraded at my startup, Blocksource), seems appropriate here: reviews can be rewarded in an incentive compatible way (I do not get into details here, for brevity).
As a side-note, let me tell you a “dirty little secret”: as the peer-review system typically does not allow multiple submissions, people try out high ranking journals (I know, shocking!) before submitting to their target journal. As a result, higher ranked journals waste resources reviewing papers that clearly have no chance of being accepted (leading to the emergence of the infamous process of “desk-rejection”, where only the editors waste their time rather than harnessing reviewers as well). Worse, lower ranked journals do not even directly benefit from this effort, as they don’t observe the reports given by higher-ranked journals. Sometimes, lower ranked journals “encourage” researchers to submit reports received in previous journals, but there is really no incentive to do so unless the previous report was positive (in which case, the researcher may disclose it voluntarily anyway). A blockchain-based peer-review system can then also provide transparency by recording previous peer-review reports. Admittedly, this may backfire, as researchers may become more selective in order to avoid bad reviews, but this is also a good thing that will crowd out bad or incomplete projects.
Another clearly useful feature that a blockchain-based peer-review system can provide is a time-stamp, proving who was the first to register an idea. This partially exists in a centralized manner, e.g. in the form of an experiments registry such as the one managed by the American Economic Association, but for a registry with the purpose of proving who was first - a decentralized system seems far better. Among else, a centralized system raises the concern of manipulation of how was first, which beats the purpose.
The second aspect relates to how experiments are conducted, and is best illustrated with an example. Suppose there is a fictitious researcher, let’s call him “Manny Pulator”. Manny has a hypothesis: people who read instructions written in a blue font are more calm than those who read instructions written in red, but less calm that those who read instructions in yellow. He designs an experiment, where he invites 240 subjects to a decision-making lab and conducts six experimental sessions. In each session, only one treatment (blue, red, yellow) is implemented, so that in the end he has 80 observations from each group. Manny looks at the data and is disappointed: the effect he thought was there is statistically insignificant. However, the hero of our story wants to get tenure at all costs, so he is thinking about behaving unethically and has three ideas on how to do that:
1) Manny can play around with the numbers, by taking the excel file generated by the program and change the data directly.
2) Manny can decide to drop the data from half of the sessions and write the paper as though he only used 120 subjects.
3) Manny can drop the entire yellow group and write the paper as though he only had two groups.
Unethical idea (1) is somewhat prevented already today, as experimental software usually generates a “read-only” excel file. However, this seems easy enough to get around, as the data is not really immutable.
Unethical ideas (2) and (3) are far harder to detect, and are a known problem also in industrial research (e.g. companies may hide some studies they did and present only those which support a positive effect of their product).
All three ideas can be prevented using blockchain technology: if data was directly recorded on-chain, our sneaky Manny could not execute any of his vicious plans, as all sessions and inputs are recorded in real time on-chain.
So why is no one constructing a blockchain-based system to record experimental data? There are several possible reasons. First, researchers and their institutions may have little incentive to propose such a system on their own, for obvious reasons. Second, journal editors do not suffer directly if it turns out that a researcher manipulated his data – as they can always claim they had no way of knowing. A system that prevents manipulation will require journals to exert effort in contrasting submitted files and blockchain records, which is undesirable for the journals. Third, adopting a blockchain-based system may be costly and difficult to implement, in particularly as the technology is still fairly new. Fourth, it is unclear whether one can make a large profit by privately constructing a system for experiments: one would need a paying customer and universities are unlikely to voluntarily purchase a system unless it is dictated by a regulator.
It is too early to say whether academia will go down the blockchain route – but it is clear that this should at least be given some serious thought.
Comments