Many people notice that scientists (especially in academia) are a socially awkward bunch. Sure, we like to party just like the rest of them, we like to go out to the pub to drink with friends, but let's face it, we are not seen as particularly charming by the rest of the population (think Igor from Frankenstein). So I got to thinking, why is it that scientists are so gloomy? Is it genetic or are there some environmental factors? Lets ask Google.
If you Google design and scientists, the top hits deal with intelligent design. Personally, I think that the somewhat gauche nature of the scientific research staff can be attributed to the lack of real artistic design in our profession. Just think of the offices where you sit, the instruments that you use and the software that you interact with. All of it looks like it was done as a throwback to the grey, square communist regimes of mass-production. Square lines, grey washed out colors and most of all, obtuse user interface design.
The reasons are probably complicated, historical and convoluted, but I can think of at least two explanations that make sense. First, scientists are not taught the importance of design. The emphasis in school is on facts and formulas not necessarily on artistic presentation. Another is that science has never been a consumer business. Internally, scientists have no say in what they choose to solve their problems as this responsibility is usually hoisted on the managers and externally, science is sold for content, not necessarily for presentation.
So how do we solve this? We create products for scientists that have well-thought out, striking design. And then we empower them to make their own decisions. We think that after we're done with them, scientists will become friendly, fluffy and charming.
Tuesday, May 26, 2009
Monday, May 25, 2009
Dynamics in scientific teams and collaborations
The massive data output of new experimental techniques changes the way science is done in academia. First, more data generated necessitates larger research teams. A diverse suite of expertise is needed to run experiments to follow up on hypotheses as well as to analyze and interpret the results. The trend is increasingly moving away from the classic one student-one advisor model to large interdisciplinary collaborations across departments, universities and countries.
Figure 2. Increasing number of authors over time on scientific publication (from Börner K, Maru JT, Goldstone RL. “The simultaneous evolution of author and paper networks.” Proc Natl Acad Sci USA. 101:5266–5273.)
Universities are struggling with providing the necessary IT infrastructure for management and analysis of data. Without the necessary tools, the usual attempts to set up academic collaborations often fall short of the stated goals or dissipate. When collaborations do happen, they often require ad-hoc solutions by informatics teams forcing graduate students or post-docs to administer servers and file-systems. This creates a de-facto overhead on each with significant inefficiencies. It also causes inequalities between research institutions that invest in high-performance computing infrastructure and those that do not.
Team science is increasingly composed of co-authors often located at different universities. These “virtual communities of scholars” produce higher impact work than solo scientists as described in (Vogeli C, Yucel R, Bendavid E, Jones LM, Anderson MS, Louis KS, Campbell EG. “Data withholding and the next generation of scientists: results of a national survey.” Acad Med. 2006 Feb; 81(2):128-36.).
With increasing investment in science from emerging economic regions, the collaborations are often spread across not only countries, but also continents. Lack of a unified collaboration workspace discourages collaboration and results in duplication of effort and loss of expertise.
So what is the solution to all these problems? We need a unified platform to allow scientists to bypass the needless administration of servers and storage, rights and privileges and other mundane tasks. In the age of internet and the flat world, we can come up with a solution tailored specifically for scientists that can adequately address these challenges. Orwik is just that platform - we hope to show that to you soon.
Figure 2. Increasing number of authors over time on scientific publication (from Börner K, Maru JT, Goldstone RL. “The simultaneous evolution of author and paper networks.” Proc Natl Acad Sci USA. 101:5266–5273.)
Universities are struggling with providing the necessary IT infrastructure for management and analysis of data. Without the necessary tools, the usual attempts to set up academic collaborations often fall short of the stated goals or dissipate. When collaborations do happen, they often require ad-hoc solutions by informatics teams forcing graduate students or post-docs to administer servers and file-systems. This creates a de-facto overhead on each with significant inefficiencies. It also causes inequalities between research institutions that invest in high-performance computing infrastructure and those that do not.
Team science is increasingly composed of co-authors often located at different universities. These “virtual communities of scholars” produce higher impact work than solo scientists as described in (Vogeli C, Yucel R, Bendavid E, Jones LM, Anderson MS, Louis KS, Campbell EG. “Data withholding and the next generation of scientists: results of a national survey.” Acad Med. 2006 Feb; 81(2):128-36.).
With increasing investment in science from emerging economic regions, the collaborations are often spread across not only countries, but also continents. Lack of a unified collaboration workspace discourages collaboration and results in duplication of effort and loss of expertise.
So what is the solution to all these problems? We need a unified platform to allow scientists to bypass the needless administration of servers and storage, rights and privileges and other mundane tasks. In the age of internet and the flat world, we can come up with a solution tailored specifically for scientists that can adequately address these challenges. Orwik is just that platform - we hope to show that to you soon.
Monday, May 18, 2009
Is Biology like Physics?
First, I would like to welcome Shanni Soyblum to our ranks of regular bloggers. Shanni is a very talented post-doc and has experience way ahead of her years with organizing collaborations and leading multi-disciplinary projects. We hope that Soyblum's blogs will become a staple for http://orwik.blogspot.com. In honor of Shanni, I will devote my next blog to the subject of collaboration as well.
Anecdotally, we've seen collaborations increase in academia and especially so in the traditionally insular world of biology. Collaborations are encouraged at all levels from departments to NIH. In one of the oldest institutions, a particularly iconoclastic university on the east-coast, the writing is one the wall. The old fiefdoms are breaking down and whether by choice or out of necessity, the faculty are starting to collaborate between each other. They need each other's expertise to solve problems and they find that they can share resources and increase their chances in getting funding.
There are two sets of evidence for increases in collaborative science. First, interdisciplinary and translational institutes are popping up faster than fungi in biowaste, after springtime rain. In Boston alone, there are now 4 huge (100M+) institutes devoted to collaborations between engineers, physicians and scientists, of which 3 were started in the last couple of years. Second, this can be seen most notably in the average number of authors per paper which has seen a threefold increase from 1.5 in 1950 to 4.5 in 2008.
Fig 1. Number of authors per publication in each year after 1950. The average number of authors per publication has increased threefold in the last fifty years.
So why are people collaborating more? Certainly some of it can be explained by digitization of biology which implies involvement from mathematicians, computer scientists and even an odd physicist that escaped from the researvation. Are these physicists going to find what they're looking for or is biology also becoming "big collaborative science" like physics?
There are certainly parallels between the history of physics and what's going on in biology now. Just like in physics, data collection is playing a larger role in biology. Data centric research tends to imply the need for collaborations between engineers and scientists. However, there are also significant differences in that biology is still very decentralized and the number of open questions is still larger than the number of groups and institutes. As biology matures as a science, the question remains : Will it become like physics with a few very large groups solving very fundamental problems or like chemistry where most of the interesting problems remain in applications and engineering?
Anecdotally, we've seen collaborations increase in academia and especially so in the traditionally insular world of biology. Collaborations are encouraged at all levels from departments to NIH. In one of the oldest institutions, a particularly iconoclastic university on the east-coast, the writing is one the wall. The old fiefdoms are breaking down and whether by choice or out of necessity, the faculty are starting to collaborate between each other. They need each other's expertise to solve problems and they find that they can share resources and increase their chances in getting funding.
There are two sets of evidence for increases in collaborative science. First, interdisciplinary and translational institutes are popping up faster than fungi in biowaste, after springtime rain. In Boston alone, there are now 4 huge (100M+) institutes devoted to collaborations between engineers, physicians and scientists, of which 3 were started in the last couple of years. Second, this can be seen most notably in the average number of authors per paper which has seen a threefold increase from 1.5 in 1950 to 4.5 in 2008.
Fig 1. Number of authors per publication in each year after 1950. The average number of authors per publication has increased threefold in the last fifty years. So why are people collaborating more? Certainly some of it can be explained by digitization of biology which implies involvement from mathematicians, computer scientists and even an odd physicist that escaped from the researvation. Are these physicists going to find what they're looking for or is biology also becoming "big collaborative science" like physics?
There are certainly parallels between the history of physics and what's going on in biology now. Just like in physics, data collection is playing a larger role in biology. Data centric research tends to imply the need for collaborations between engineers and scientists. However, there are also significant differences in that biology is still very decentralized and the number of open questions is still larger than the number of groups and institutes. As biology matures as a science, the question remains : Will it become like physics with a few very large groups solving very fundamental problems or like chemistry where most of the interesting problems remain in applications and engineering?
Sunday, May 17, 2009
A Different Perspective
Today I don a new hat as guest science blogger for the Orwik Blog. I’ve been watching Orwik’s progress closely, and paying attention to how Orwik is trying to address the way collaborative science is conducted. Who am I and why do I care? I’m the project manager of an interdisciplinary research team at a Canadian university- 12 faculty members spanning five departments and supporting 20+ grad students, postdocs and technicians. I’m also a postdoc working within the project with my own set of research problems, thinking about sharing data, publishing and where I stand in the scientific community. I’ll be posting from the trenches about everything from university politics and research funding to being a postdoc, collaborating and rallying the troops to move our project forward.
In my position, I often find myself carefully trying to be both diplomatic and assertive, believing in my perhaps idealistic way that this is the best approach for making progress. However, last week’s events make me question whether it’s better to be outright aggressive at times, demanding change rather than cautiously trying to negotiate it. Recent government cuts to university budgets left our project in a lurch earlier this month when our project budget was cut at the last minute by 13%. We scrambled to adjust our budget, eliminating two personnel positions, all travel and a conference we were planning to host. We wrote a carefully worded letter to the VP of Research explaining the effects of the last-minute cut, but generally accepted that given the current economic climate, we were lucky to even have funding. In contrast, members of a similarly funded project with a similar budget cut called the office “kicking and screaming” (their words, not mine) and demanded the return of their funds, which they subsequently received. It may be too late for us to employ this strategy (though I hear rumors that the government is quietly reinstating some of the university funds), but it makes me keenly aware that I have a lot to learn about dealing with university politics.
In my position, I often find myself carefully trying to be both diplomatic and assertive, believing in my perhaps idealistic way that this is the best approach for making progress. However, last week’s events make me question whether it’s better to be outright aggressive at times, demanding change rather than cautiously trying to negotiate it. Recent government cuts to university budgets left our project in a lurch earlier this month when our project budget was cut at the last minute by 13%. We scrambled to adjust our budget, eliminating two personnel positions, all travel and a conference we were planning to host. We wrote a carefully worded letter to the VP of Research explaining the effects of the last-minute cut, but generally accepted that given the current economic climate, we were lucky to even have funding. In contrast, members of a similarly funded project with a similar budget cut called the office “kicking and screaming” (their words, not mine) and demanded the return of their funds, which they subsequently received. It may be too late for us to employ this strategy (though I hear rumors that the government is quietly reinstating some of the university funds), but it makes me keenly aware that I have a lot to learn about dealing with university politics.
Monday, May 11, 2009
Transparency in Exponential Science
I am sure that all the readers of this blog will not find it a surprise that the amount of information in biomedical sciences is increasing exponentially. Every kind of information is increasing whether it be sequence data, structure data, SNP data or simply the number of journal articles. We wrote one article about this on this blog. This is a followup to that article given the recent announcement published in the Scientist that Merck and Elsevier have published 7 fake journals. These journals were not really "fake", but instead failed to disclose 
their affiliation with the pharmaceutical giant.
Number of publications indexed by Pubmed over the last 50 years.
In fact, I think that most of the comments to this article and the article itself misses the point. We should not care who sponsored the research as long as that research is high quality and adds some important insight into the world scientific knowledgebase. However, instead of focusing on these issues we are forced to look at anciliary evidence such as sponsorship and affiliation. The reason is that our feedback on the quality of the research is too slow.
We would have to wait for years until a clear picture was established on the substance of the manuscripts. Feedback requires not only that other people write their own articles, but that they undergo review and get published (go through the editorial cycle). This process is not only slow, but also very painful so few academics are willing to go through it if they get nothing in return themselves. For well cited papers, a recent article estimated that it takes on average 10 years until signigficant numbers of citations develops.
This excrutiatingly slow pace of feedback is the main culrpit for lack of transparency in science. If I can't reliably determine the accuracy of what I'm reading based on the substance of the work, I am forced to use other characteristics such as : trust for the person writing it, or respect for the institution where the authors are from, or devotion to the brand of the article. However, as the Elsevier debacle showed, these secondary characteristics are subject to corruption and are highly unreliable.
The solution to scientific transparency has been found many times over, the last time in the form of Wikipedia. It involves creating a system where the community is involved in quickly and accurately responding to new publications and data.
their affiliation with the pharmaceutical giant.Number of publications indexed by Pubmed over the last 50 years.
In fact, I think that most of the comments to this article and the article itself misses the point. We should not care who sponsored the research as long as that research is high quality and adds some important insight into the world scientific knowledgebase. However, instead of focusing on these issues we are forced to look at anciliary evidence such as sponsorship and affiliation. The reason is that our feedback on the quality of the research is too slow.
We would have to wait for years until a clear picture was established on the substance of the manuscripts. Feedback requires not only that other people write their own articles, but that they undergo review and get published (go through the editorial cycle). This process is not only slow, but also very painful so few academics are willing to go through it if they get nothing in return themselves. For well cited papers, a recent article estimated that it takes on average 10 years until signigficant numbers of citations develops.
This excrutiatingly slow pace of feedback is the main culrpit for lack of transparency in science. If I can't reliably determine the accuracy of what I'm reading based on the substance of the work, I am forced to use other characteristics such as : trust for the person writing it, or respect for the institution where the authors are from, or devotion to the brand of the article. However, as the Elsevier debacle showed, these secondary characteristics are subject to corruption and are highly unreliable.
The solution to scientific transparency has been found many times over, the last time in the form of Wikipedia. It involves creating a system where the community is involved in quickly and accurately responding to new publications and data.
Sunday, May 10, 2009
Is Open Access like Modern Ballet?
When going to modern ballet, you see the stark contrasts between the traditional and the new. The crowd, which is mostly over 50, is gathered to look at twenty year old, long-limbed ballerinas dancing provocatively in tight-fitting leotards. The dancers, alternatively doing long, athletic Grand Jete and thrusting boogie-woogie-style do it all to music written two centuries ago to stories written by writers that died in duels. As anachronistic as it seems, the hall is full and the audience awaits with baited breath for the next scene. When everyone comes out for intermission, the purists grumble about how the "original was better". Yet, no one leaves and all shuffle back as the lights dim for the second act. The impact of modern dance on ballet is clear and profound. This year, that one modern ballet company will do more shows in our city than all the local ballet troupes combined.The picture is from a dress rehersal of Boris Eifman's Cassandra
It seems to me that Open Access Publishing, especially the type which is enabled by the new realities of the internet, resembles modern ballet. The usually conservative scientists reluctantly embracing technology discovered and popularized by a bunch of twenty somethings. The stodgy publishers turn to the young technocrats to show them the way science publishing will look in the future. While everyone is transfixed, waiting for the arrival of the new reality in science communication, the purists claim that publishing should stay the way it was.
In fact, the purists of both ballet and science publishing have nothing to fear. Modern ballet has not transplanted traditional ballet. Open access has been around for more than a decade in physics in the form of Arxiv, and yet Physical Review has more submissions than ever. That being said, I applaud the courage of the entrepreneur who injects modernity into traditional disciplines, whether they be dance or science.
Saturday, May 9, 2009
The rough cut
Each web startup has to face the harsh reality of users. We have started to explore this new reality for ourselves. Let me tell you, it is painful. While the process of building something from scratch is fundamental, the process of using it is specific. Sometimes the two are at odds and their reconciliation is integral to the survival of the product. Let me outline what I mean with a couple of examples.
A really cool company that I've been following lately is Local Motors. These guys are trying to build cars even though Toyota just announced that they've lost over 7 billion dollars this quarter. So how can a couple of guys in a garage hope to build cars that are cheap enough so that the company is profitable, but cool enough that people want to buy them? Malcolm Gladwell recently wrote an article in the New Yorker that outlines the advantages of changing the established strategies for insurgents against entrenched, larger players. While GM and Ford are cutting costs by producing cars that all look alike, Local Motors are trying decrease costs by crowdsourcing design. In this case, playing outside the established traditions, LM can fundamentally change the way cars are built.
Another case where design has had a significant impact is one of my favorite companies: 37 signals. These guys took an age-old problem of collaboration on the web and applied minimalist design to make it dead simple. They were also the original developers of Rails, an open-source platform very popular with web-developers. As a result, they have been enjoying enormous success both in transforming the way people think about web software and in how people approach project management.
There are two things that unify these two example. First, when technology becomes commoditized, design becomes the value-added competitive advantage. Second, there is significant value in adding open-source as both a marketing tool and as a way to get community involvement.
Now that we've built the technology, we are working with our beta testers to chisel out the rough cut. Focusing on the needs of our customers will help us to design the best research life-cycle management software on the web. The road ahead is long, but we're very excited to be on it.
A really cool company that I've been following lately is Local Motors. These guys are trying to build cars even though Toyota just announced that they've lost over 7 billion dollars this quarter. So how can a couple of guys in a garage hope to build cars that are cheap enough so that the company is profitable, but cool enough that people want to buy them? Malcolm Gladwell recently wrote an article in the New Yorker that outlines the advantages of changing the established strategies for insurgents against entrenched, larger players. While GM and Ford are cutting costs by producing cars that all look alike, Local Motors are trying decrease costs by crowdsourcing design. In this case, playing outside the established traditions, LM can fundamentally change the way cars are built.
Another case where design has had a significant impact is one of my favorite companies: 37 signals. These guys took an age-old problem of collaboration on the web and applied minimalist design to make it dead simple. They were also the original developers of Rails, an open-source platform very popular with web-developers. As a result, they have been enjoying enormous success both in transforming the way people think about web software and in how people approach project management.
There are two things that unify these two example. First, when technology becomes commoditized, design becomes the value-added competitive advantage. Second, there is significant value in adding open-source as both a marketing tool and as a way to get community involvement.
Now that we've built the technology, we are working with our beta testers to chisel out the rough cut. Focusing on the needs of our customers will help us to design the best research life-cycle management software on the web. The road ahead is long, but we're very excited to be on it.
Sunday, May 3, 2009
What it means to be Bold in Science Publishing
Science publishers such as Elsevier and Springer are not usually known for their forward looking technology and business practices. However, with the near demise of their counterparts in the newspaper business, many are seeing the writing on the wall. Clearly, there are several fundamental forces that have already significantly affected general publishing and are going to put significant amount of pressure on scientific publishers' revenue streams in the coming years.
The rise of the internet allows extremely cheap publishing and distribution of information. In the real world, these forces have led to the rise of Blogs, Wikis and other publishing platforms that have decimated the newspapers by fragmenting the marketplace and significantly decreasing readership of general purpose publications such as the New York times. The information explosion precipitated by decreases in publishing cost has put a significant premium on finding relevant information. This results in the emergence of portals like Google and Yahoo whose sole business revolves around separating junk. There are also science publisher specific changes such as digitization of manuscripts into .pdf formats, the subsequent shift to getting manuscripts online, and the difficulties in extracting semantic information from scientific papers which makes monetization through online ad advertising extremely challenging.
The fact that their constituency is also very conservative has allowed science publishers to fare better than their general-purpose counterparts. However, with the massive failure of print newspapers, even the scientific publishers are taking notice and taking action. Nature, for example, has been coming out with a steady stream of online me-too services such as the reference manager Connotea and the Nature Network (social network for scientists). Even Elsevier has started a new Grand Challenge where they challenge participants to show them the way that publishing will look in the future. The winners this year are a team from EMBL who came up with an markup dictionary. The tool (called Reflect) has a direct parallel in the real world with Answers.com.
The bubble years have seen the rise and subsequent fall of many scientific portals, lab-notebooks and other online services. These have never garnered much popularity with the scientists and all have fallen by the wayside. Judging from the presenters at the BioIT World Expo, this is the beginning of Science Web 2.0 with many new offerings in the same arenas of portals, lab notebooks and aggregators.
I am sure that most of these companies have studied the failure of their forebearers. So why have the initiatives been met with limited success in the academic world? I think that no one really knows the answer to that question fully. I am sure that many believe that science has changed in the last decade and that the time is right now. Another part of it is that academia is not-for-profit so it moves generally slower in adopting new technologies.
However, I believe that the problem is much more transcendent and fundamental. The fact is that nobody has found a solution that is specific to science and addresses the problems faced by scientists in their day-to-day lives. The services offered by these companies and publishers have been mostly rip-offs from other successful businesses, which have very different and specific dynamics that are not present in science. Thus, I submit that the future of science publishing, like its origins, will be built by scientists for scientists.
The rise of the internet allows extremely cheap publishing and distribution of information. In the real world, these forces have led to the rise of Blogs, Wikis and other publishing platforms that have decimated the newspapers by fragmenting the marketplace and significantly decreasing readership of general purpose publications such as the New York times. The information explosion precipitated by decreases in publishing cost has put a significant premium on finding relevant information. This results in the emergence of portals like Google and Yahoo whose sole business revolves around separating junk. There are also science publisher specific changes such as digitization of manuscripts into .pdf formats, the subsequent shift to getting manuscripts online, and the difficulties in extracting semantic information from scientific papers which makes monetization through online ad advertising extremely challenging.
The fact that their constituency is also very conservative has allowed science publishers to fare better than their general-purpose counterparts. However, with the massive failure of print newspapers, even the scientific publishers are taking notice and taking action. Nature, for example, has been coming out with a steady stream of online me-too services such as the reference manager Connotea and the Nature Network (social network for scientists). Even Elsevier has started a new Grand Challenge where they challenge participants to show them the way that publishing will look in the future. The winners this year are a team from EMBL who came up with an markup dictionary. The tool (called Reflect) has a direct parallel in the real world with Answers.com.
The bubble years have seen the rise and subsequent fall of many scientific portals, lab-notebooks and other online services. These have never garnered much popularity with the scientists and all have fallen by the wayside. Judging from the presenters at the BioIT World Expo, this is the beginning of Science Web 2.0 with many new offerings in the same arenas of portals, lab notebooks and aggregators.
I am sure that most of these companies have studied the failure of their forebearers. So why have the initiatives been met with limited success in the academic world? I think that no one really knows the answer to that question fully. I am sure that many believe that science has changed in the last decade and that the time is right now. Another part of it is that academia is not-for-profit so it moves generally slower in adopting new technologies.
However, I believe that the problem is much more transcendent and fundamental. The fact is that nobody has found a solution that is specific to science and addresses the problems faced by scientists in their day-to-day lives. The services offered by these companies and publishers have been mostly rip-offs from other successful businesses, which have very different and specific dynamics that are not present in science. Thus, I submit that the future of science publishing, like its origins, will be built by scientists for scientists.
Wednesday, April 29, 2009
Franklin Award
With tax season and grant season, it's been a harrowing couple of weeks for us here at Orwik central. Lots of things have happened and I will try to catch everyone up on the latest developments. Let's start with the Benjamin Franklin award. This is the award given annually to the person who contributes the most to the progression of the Open Access movement. Given at the BioIT World Conference and Expo, this year's recipient was our own Phil Bourne. Other recipients of the prize have been some notable advocates of Open Access including Mike Eisen and Michael Ashburner.
We were talking with Phil about Orwik after the presentation, and he said that in his dealings with publishers and pharmaceuticals, he found that many are embracing the new OA movement. This is also supported by the recent spate of news detailing plans to publish pre-competitive data by Pharmaceutical Industry giants. One such effort is Sage, spearheaded by Eric Shadt and Merck. There is a very good review of Sage in a recent blog posting from molecular philosophy. While the issue of OA is clearly at the forefront of many executives' minds, the execution and the value of these initiatives are still under question. Like the review of Sage suggests, an outline of the long-term goals for data publication would go a long way towards establishing the value of open access to for-profit corporations.
We were talking with Phil about Orwik after the presentation, and he said that in his dealings with publishers and pharmaceuticals, he found that many are embracing the new OA movement. This is also supported by the recent spate of news detailing plans to publish pre-competitive data by Pharmaceutical Industry giants. One such effort is Sage, spearheaded by Eric Shadt and Merck. There is a very good review of Sage in a recent blog posting from molecular philosophy. While the issue of OA is clearly at the forefront of many executives' minds, the execution and the value of these initiatives are still under question. Like the review of Sage suggests, an outline of the long-term goals for data publication would go a long way towards establishing the value of open access to for-profit corporations.
Thursday, April 9, 2009
Evidence of Personal Incentives in Open Access
I recently found a couple of interesting studies that compare the citation rate for articles which were Open Access to those that were not. I think that it is well worth the read and at least a cursory look for those interested in the subject.
First, there is a very interesting article outlining the advantages of Open Access in garnering more citations. Gunther Eysenbach builds on a more comprehensive study in computer science and physics. Both studies show that OA articles get up to twice as many citations!
I think that the evidence is now undisputable. Making science available for free increases its visibility and usefulness and garners appropriate rewards to the author.
First, there is a very interesting article outlining the advantages of Open Access in garnering more citations. Gunther Eysenbach builds on a more comprehensive study in computer science and physics. Both studies show that OA articles get up to twice as many citations!
I think that the evidence is now undisputable. Making science available for free increases its visibility and usefulness and garners appropriate rewards to the author.
Why We Want to Share Data but Don't
In a recent opinion piece in the Scientist, Steven Wiley argues that the issue of data sharing has been around for a long time and that government agencies have tried various efforts to solve that issue but failed. As evidence he points out that AfCS has been cited in only a handful of publications. He concludes that the nature of science prevents people from sharing and using each other's data.
It's worth noting that this posting generated 3 replies, all of which are negative and argue against the opinion. As the replies to the piece indicate and initiatives such asAfCS, CaBIG and BioGrid make clear, we REALLY want to share data. So is Wiley totally wrong? And if not, where is the disconnect?
First, Wiley may be incorrect in choosing the metric of success. As he points out himself, the AfCS database recieves 100,000 weekly page-views. This suggests that AfCS is significantly more popular than top-tier journals like PLOS, PNAS, and CELL (44,000 MONTHLY visits). Clearly, people are using the data.
Second, AFCS may not be entirely representative. In fact, over the last several decades, data-rich publications such as genome projects and databases have garnered the most citations. In fact, the authors of genome publications (Lander, Venter, etc.) and database publications (Bourne, Lipman) have become some of the most widely cited authors.
Finally, assuming that some people still don't share data despite the clear benefits to their careers (papers that publish data get up to 80% of citations), what could be the reason? The reason I submit for this lack of sharing is part legacy, part execution. People are not used to sharing data. Data publication has become a topical issue only recently when the amount of data exceeded the limits of publishing, which is only recently in most biological disciplines. People still find it hard to publish their data. Either you have to create a website or you have to find a database to stick that data into.
There is a push towards making data available from institutions and significant evidence that it benefits both the society at large and the generator of the data. Change always takes time and changing cultures take the longest. However, I am very heartened by the vector, which I think points clearly in the right direction.
Barak Shahen
It's worth noting that this posting generated 3 replies, all of which are negative and argue against the opinion. As the replies to the piece indicate and initiatives such asAfCS, CaBIG and BioGrid make clear, we REALLY want to share data. So is Wiley totally wrong? And if not, where is the disconnect?
First, Wiley may be incorrect in choosing the metric of success. As he points out himself, the AfCS database recieves 100,000 weekly page-views. This suggests that AfCS is significantly more popular than top-tier journals like PLOS, PNAS, and CELL (44,000 MONTHLY visits). Clearly, people are using the data.
Second, AFCS may not be entirely representative. In fact, over the last several decades, data-rich publications such as genome projects and databases have garnered the most citations. In fact, the authors of genome publications (Lander, Venter, etc.) and database publications (Bourne, Lipman) have become some of the most widely cited authors.
Finally, assuming that some people still don't share data despite the clear benefits to their careers (papers that publish data get up to 80% of citations), what could be the reason? The reason I submit for this lack of sharing is part legacy, part execution. People are not used to sharing data. Data publication has become a topical issue only recently when the amount of data exceeded the limits of publishing, which is only recently in most biological disciplines. People still find it hard to publish their data. Either you have to create a website or you have to find a database to stick that data into.
There is a push towards making data available from institutions and significant evidence that it benefits both the society at large and the generator of the data. Change always takes time and changing cultures take the longest. However, I am very heartened by the vector, which I think points clearly in the right direction.
Barak Shahen
Sunday, April 5, 2009
Deflationary Pressures on Science
In this world of economic crises and slacking demand, there is another Exponential Economy that is similarly in dire straits. The scientific knowledge of the world has been growing at an exponential rate with average compound growth of 4.8% for the last 50 years.
That's on the supply side. So what about demand? Who needs 10 Million Scientific Articles and what do they need it for? Most importantly, is the world of scientific knowledge undergoing a prolonged deflationary cycle? Perhaps the rise of Open Access Publishing is, in fact, the transition where producers of scientific knowledge start paying consumers of that knowledge due to over-supply and very limited demand.
I've thought a lot about these questions but have not come to any conclusions. If scientific ideas are becoming commoditized, this may herald a complete change in the way we communicate and think about science, invention and innovation. Under this model, first to publish and quantity would trump quality and brand. Do as much science as possible as quickly as possible and tell others that you did it.
Clearly, science has not entirely reached that point, but individual disciplines like physics and mathematics have shown some signs that speed and quantity are important in establishing your scientific role. It is worth noting that areas which have been around longer have seen more profound movements in that direction. Experimental sciences have been slower to adopt this strategy, perhaps because of the inherent limit on how quickly one can complete experiments in the lab.
Some visionary thinkers in the areas of science and business have clearly thought about these issues as well. For example, Nathan Myhrvold's company Intellectual Ventures collects patents as a way to monetize those inventions in the future. IV claims that it is impossible to tell which ideas will pan out in the end, so creating and collecting as many as possible is the best strategy. One of our own visionary investors, George Whitesides, has a policy of publishing all results; because if it wasn't published, it wasn't done. Its worth noting that Whitesides is the second most cited chemist in the world.
So is it really true that knowledge and invention are becoming a commodity? I think that this isn't necessarily so simple. If this was true, pharmaceutical companies would not be reeling from empty drug pipelines despite pouring in billions of dollars into their R+D. In a recent article, Nathan Myhrvold says that the most important discoveries are made at the fringes. This is consistent with the notion that it is in those untouched areas where ideas are truly 'new'. Following this line of thought, perhaps we should be thinking more about creating tools to improve the efficiency of identifying the most promising research at the fringes.
Barak Shahen
That's on the supply side. So what about demand? Who needs 10 Million Scientific Articles and what do they need it for? Most importantly, is the world of scientific knowledge undergoing a prolonged deflationary cycle? Perhaps the rise of Open Access Publishing is, in fact, the transition where producers of scientific knowledge start paying consumers of that knowledge due to over-supply and very limited demand.
I've thought a lot about these questions but have not come to any conclusions. If scientific ideas are becoming commoditized, this may herald a complete change in the way we communicate and think about science, invention and innovation. Under this model, first to publish and quantity would trump quality and brand. Do as much science as possible as quickly as possible and tell others that you did it.
Clearly, science has not entirely reached that point, but individual disciplines like physics and mathematics have shown some signs that speed and quantity are important in establishing your scientific role. It is worth noting that areas which have been around longer have seen more profound movements in that direction. Experimental sciences have been slower to adopt this strategy, perhaps because of the inherent limit on how quickly one can complete experiments in the lab.
Some visionary thinkers in the areas of science and business have clearly thought about these issues as well. For example, Nathan Myhrvold's company Intellectual Ventures collects patents as a way to monetize those inventions in the future. IV claims that it is impossible to tell which ideas will pan out in the end, so creating and collecting as many as possible is the best strategy. One of our own visionary investors, George Whitesides, has a policy of publishing all results; because if it wasn't published, it wasn't done. Its worth noting that Whitesides is the second most cited chemist in the world.
So is it really true that knowledge and invention are becoming a commodity? I think that this isn't necessarily so simple. If this was true, pharmaceutical companies would not be reeling from empty drug pipelines despite pouring in billions of dollars into their R+D. In a recent article, Nathan Myhrvold says that the most important discoveries are made at the fringes. This is consistent with the notion that it is in those untouched areas where ideas are truly 'new'. Following this line of thought, perhaps we should be thinking more about creating tools to improve the efficiency of identifying the most promising research at the fringes.
Barak Shahen
Saturday, April 4, 2009
I am not a Scientist, I am a Number
In a recent article, our good friends Phil Bourne and J. Lynn Fink wrote about a new system championed by Thomson Reuters. The system is based on the OpenID system and is cleverly dubbed ResearcherID. The system ties in your profile and your published materials with a number thus eliminating the ambiguity of human names like Smith.
In principle, this is not a bad idea but as Bourne and Fink point out, there are many ifs. Other identification efforts championed on the web have not taken off. Passport from Microsoft was one of the more prominent ones. I think that Thomson Reuters would benefit from understanding the reasons behind the failure of these systems.
Here's my take on this. First, I don't want to be a number. I want to be a person. I am flesh and blood and personality and a number is none of these. Second, I don't really care about making it easier to disambiguate my name : Barak Shahen. How many of us are out there? In fact, how many scientists have truly ambiguous names? On the other hand, these systems can only work if there is wide adoption. Finally, this only makes sense for me if the publishers, universities and other organizations make ResearcherID easy and transparent. What are the chances of that happening any time soon?
That being said, OpenID is trying to solve an important problem. We at Orwik have solved the problem of disambiguation in a different way but to find out details, you'll have to stay tuned here.
Barak Shahen
P.S. Bourne and Fink propose a different and intriguing benefit of ResearcherID: the possibility of quantifying research impact based on a combination of metrics other than the citation index. I think that this is a great idea and goes directly to some of the things I blogged about in my previous post.
In principle, this is not a bad idea but as Bourne and Fink point out, there are many ifs. Other identification efforts championed on the web have not taken off. Passport from Microsoft was one of the more prominent ones. I think that Thomson Reuters would benefit from understanding the reasons behind the failure of these systems.
Here's my take on this. First, I don't want to be a number. I want to be a person. I am flesh and blood and personality and a number is none of these. Second, I don't really care about making it easier to disambiguate my name : Barak Shahen. How many of us are out there? In fact, how many scientists have truly ambiguous names? On the other hand, these systems can only work if there is wide adoption. Finally, this only makes sense for me if the publishers, universities and other organizations make ResearcherID easy and transparent. What are the chances of that happening any time soon?
That being said, OpenID is trying to solve an important problem. We at Orwik have solved the problem of disambiguation in a different way but to find out details, you'll have to stay tuned here.
Barak Shahen
P.S. Bourne and Fink propose a different and intriguing benefit of ResearcherID: the possibility of quantifying research impact based on a combination of metrics other than the citation index. I think that this is a great idea and goes directly to some of the things I blogged about in my previous post.
Valuing Information in Science
One of the hardest things to do in science is put a value on information. So what prevents us from doing the same thing with science as we do with our latte's every day: Coming in and saying, "This piece of data is worth X dollars." The problems stem from inefficiencies of how we communicate and distribute scientific progress.
If we were to look at academic science as a marketplace, what would we learn? Lets start with defining the players. The main products are the papers and the data. The producers are the authors and the consumers are the readers. What does this marketplace lack? Incredibly, this marketplace lacks the most basic part needed to evaluate products: the currency!
Without appropriate currency, we can not know the value of papers and scientific results. Sure, there are obviously important discoveries (like the structure of the DNA) or the seminal paper describing the semi-conservative replication of DNA by Meselson and Stahl. But, how do we know that a discovery is important? How do we know the "value" of a paper?
There are two answers that I hear most often to the above questions. First, my colleagues tell me, "I know people in my field so I know who to believe." However, as science becomes bigger and people publish more often and move around between fields with increasing frequency, this method of "guild-membership" stops working.
The other traditional answer to this has to do with citation index: a measure of how many other authors deemed your work worthy enough to cite in their own papers. The problem here is that the turnaround time is measured in years. Someone has to read your work, do a followup experiment and then go through the process of publishing. Imagine if I created a car, sold it to you, but only got payment three years later.
Both these methods of evaluating scientific work create huge inefficiencies in academic science. These inefficiencies all stem from legacy issues in scientific publishing, which was established in the 17th century. We can do better. Economic theory states that income is measured in absolute monetary supply multiplied by the speed at which money moves through the system. One way to increase the utility of knowledge is to increase the speed at which that knowledge is evaluated by the community.
Barak Shahen.
If we were to look at academic science as a marketplace, what would we learn? Lets start with defining the players. The main products are the papers and the data. The producers are the authors and the consumers are the readers. What does this marketplace lack? Incredibly, this marketplace lacks the most basic part needed to evaluate products: the currency!
Without appropriate currency, we can not know the value of papers and scientific results. Sure, there are obviously important discoveries (like the structure of the DNA) or the seminal paper describing the semi-conservative replication of DNA by Meselson and Stahl. But, how do we know that a discovery is important? How do we know the "value" of a paper?
There are two answers that I hear most often to the above questions. First, my colleagues tell me, "I know people in my field so I know who to believe." However, as science becomes bigger and people publish more often and move around between fields with increasing frequency, this method of "guild-membership" stops working.
The other traditional answer to this has to do with citation index: a measure of how many other authors deemed your work worthy enough to cite in their own papers. The problem here is that the turnaround time is measured in years. Someone has to read your work, do a followup experiment and then go through the process of publishing. Imagine if I created a car, sold it to you, but only got payment three years later.
Both these methods of evaluating scientific work create huge inefficiencies in academic science. These inefficiencies all stem from legacy issues in scientific publishing, which was established in the 17th century. We can do better. Economic theory states that income is measured in absolute monetary supply multiplied by the speed at which money moves through the system. One way to increase the utility of knowledge is to increase the speed at which that knowledge is evaluated by the community.
Barak Shahen.
Wednesday, April 1, 2009
Policy versus Practice
This is going to be a short post/vent about the recent rash of Open Access Policies brought forth by leading Universities. Harvard was one of the first Universities to declare support for an Open Access Policy. Press releases followed, such as this one here and the more sensational one about the MIT policy here. In fact, these policies cause a lot of confusion.
If we dig deeper, we find the details behind the policy in an unusually long and boring article. Let me save you the trouble of reading through the whole thing as it was painful enough for me. The skinny is that Harvard holds brown-bag lunches during which they tell you about how to comply with the NIH public access policy. I know that I'm going to go real soon because it sounds thrilling. Case in point: even Hal Abelson, who is NOT technically challenged as an MIT CS professor and the biggest proponent of the Open Access, has nothing of note in the MIT OA (DSpace) repository as of the time of writing this blog.
Herein lies the duality between Policy and Practice: talking about it and doing it. People used to tease me with the infamous phrase, "Those who can't do, teach". This is a perfect example. Instead of empowering people by giving them tools, policy wonks describe in excruciating detail the vagaries of negotiating copyright licenses and NIH compliance memos. The result is that while all this is talked and written about, nothing gets accomplished.
Now imagine that instead of writing Napster, Shawn Fanning held informational meetings on how to take floppy disks and send them to your friends via snail mail. In those meetings, Fanning would also point out that before sending the disk, music fans should send an informational memo to the copyright owner informing them that they wish to renegotiate their agreements. Revolutions (the good kind) are not started by simply telling people what they already know. Change can be achieved only by empowering millions to do what they otherwise would not be able or willing to do.
Like any good but too-quickly-popularized idea, Open Access can easily fall prey to opportunistic politicians who get up on the soapbox thinking that this will increase their popularity. Let's consider for a second that memorandas and meetings will convince scientists to submit their work to the Provost's office. What are the long-term consequences of this? What is the incentive for the University, or the Provost? Is this really the "right" way to publish? Is this the way to promote and distribute science? In the same Bloomberg article, Harold Varmus, who was one of the founding fathers of the OA movement and one of its biggest proponents, cautions that peer review and details in execution are both important in scientific publishing and science as a whole. I have to side with my man Harold here.
It's easy to vote; it's much harder to really consider the consequences of your vote; and it may be harder still to live with those consequences. Open Access is an idea that may turn into a movement, but it is still only an idea. Until then, it is easy to distort and end up with the opposite result. Scientists should consider carefully whether they may be sawing with reckless abandon the limb on which they're so precariously perched.
Barak Shahen
If we dig deeper, we find the details behind the policy in an unusually long and boring article. Let me save you the trouble of reading through the whole thing as it was painful enough for me. The skinny is that Harvard holds brown-bag lunches during which they tell you about how to comply with the NIH public access policy. I know that I'm going to go real soon because it sounds thrilling. Case in point: even Hal Abelson, who is NOT technically challenged as an MIT CS professor and the biggest proponent of the Open Access, has nothing of note in the MIT OA (DSpace) repository as of the time of writing this blog.
Herein lies the duality between Policy and Practice: talking about it and doing it. People used to tease me with the infamous phrase, "Those who can't do, teach". This is a perfect example. Instead of empowering people by giving them tools, policy wonks describe in excruciating detail the vagaries of negotiating copyright licenses and NIH compliance memos. The result is that while all this is talked and written about, nothing gets accomplished.
Now imagine that instead of writing Napster, Shawn Fanning held informational meetings on how to take floppy disks and send them to your friends via snail mail. In those meetings, Fanning would also point out that before sending the disk, music fans should send an informational memo to the copyright owner informing them that they wish to renegotiate their agreements. Revolutions (the good kind) are not started by simply telling people what they already know. Change can be achieved only by empowering millions to do what they otherwise would not be able or willing to do.
Like any good but too-quickly-popularized idea, Open Access can easily fall prey to opportunistic politicians who get up on the soapbox thinking that this will increase their popularity. Let's consider for a second that memorandas and meetings will convince scientists to submit their work to the Provost's office. What are the long-term consequences of this? What is the incentive for the University, or the Provost? Is this really the "right" way to publish? Is this the way to promote and distribute science? In the same Bloomberg article, Harold Varmus, who was one of the founding fathers of the OA movement and one of its biggest proponents, cautions that peer review and details in execution are both important in scientific publishing and science as a whole. I have to side with my man Harold here.
It's easy to vote; it's much harder to really consider the consequences of your vote; and it may be harder still to live with those consequences. Open Access is an idea that may turn into a movement, but it is still only an idea. Until then, it is easy to distort and end up with the opposite result. Scientists should consider carefully whether they may be sawing with reckless abandon the limb on which they're so precariously perched.
Barak Shahen
Tuesday, March 31, 2009
Why Open Access Advocates Don't Tell the Whole Truth
The Open Access movement is a relatively recent development where scientists argue that the papers they publish should be free for all to read. In turn, the publishers argue that they provide a useful service in the form of editing and peer-review that costs money and should be compensated. Elsevier, for example, gets compensated to the tune of 10B dollars a year.
Open Access advocates, of which I am one, use a common argument to support their cause for a free publication system. This argument usually goes like this: The work that I am publishing has already been paid for by the taxpayers and the granting institution, therefore the publishing should be free. That, of course, is bullocks and obscures the real reason why scientists want their work to be free.
Many things that the taxpayers and government pay for are not free. For example, taxpayers pay for that F16 flying sorties above Baghdad. Can I then go to Andrews and request a free joy-ride on the F16 because my taxpayer dollars paid for it? I think the government may have something to say against that. You can argue that the F16 is a "thing" and science is knowledge, but I can't look at the blueprints for the F16 either. I can't even see the results of most tests on failed materials NOT used to make that F16. Here's another interesting tidbit: very few executives or employees of for-profit companies funded through the same NIH dollars (through SBIR) want their work to be free and available in the public domain.
So what is the real reason why authors in academia want their work to be free after publication? From the point of view of the author, having the publisher charge for their work does not bring them any additional value. In fact, it is detrimental to the notoriety of that work as paid access decreases the number of people that could potentially read the publication.
So why do authors still publish in Nature, Science, Cell and other journals that do not put the manuscripts up on the web for free? The answer is simple. The authors want their cake and eat it too. They want the branding of Nature that brings attention, validation and readers. In fact, many academics are afraid that if they publish in an open access journal like BMC or PLOS, their work will be "lost" in the shuffle of millions of papers that are published each year.
Of course, the big-name journals are less than genuine as well. The real "value" that they provide is not the editing and peer-review as they often claim. The value that they provide is marketing. The publishers are paid to reject. Journals are often considered to be better if they reject more manuscripts. Rejection of everyone's manuscript but their own allows authors to feel like they belong to an exclusive club. Memberships in exclusive clubs, as many companies have proven time an again, is worth money.
I think that proper understanding and adoption of Open Access will come only after we begin a dialogue where the authors and publishers are honest about the value they extract from each other. I will argue in my following posts that, in fact, the value of the brand is far less than the cost of limiting access to your work.
Barak Shahen
Open Access advocates, of which I am one, use a common argument to support their cause for a free publication system. This argument usually goes like this: The work that I am publishing has already been paid for by the taxpayers and the granting institution, therefore the publishing should be free. That, of course, is bullocks and obscures the real reason why scientists want their work to be free.
Many things that the taxpayers and government pay for are not free. For example, taxpayers pay for that F16 flying sorties above Baghdad. Can I then go to Andrews and request a free joy-ride on the F16 because my taxpayer dollars paid for it? I think the government may have something to say against that. You can argue that the F16 is a "thing" and science is knowledge, but I can't look at the blueprints for the F16 either. I can't even see the results of most tests on failed materials NOT used to make that F16. Here's another interesting tidbit: very few executives or employees of for-profit companies funded through the same NIH dollars (through SBIR) want their work to be free and available in the public domain.
So what is the real reason why authors in academia want their work to be free after publication? From the point of view of the author, having the publisher charge for their work does not bring them any additional value. In fact, it is detrimental to the notoriety of that work as paid access decreases the number of people that could potentially read the publication.
So why do authors still publish in Nature, Science, Cell and other journals that do not put the manuscripts up on the web for free? The answer is simple. The authors want their cake and eat it too. They want the branding of Nature that brings attention, validation and readers. In fact, many academics are afraid that if they publish in an open access journal like BMC or PLOS, their work will be "lost" in the shuffle of millions of papers that are published each year.
Of course, the big-name journals are less than genuine as well. The real "value" that they provide is not the editing and peer-review as they often claim. The value that they provide is marketing. The publishers are paid to reject. Journals are often considered to be better if they reject more manuscripts. Rejection of everyone's manuscript but their own allows authors to feel like they belong to an exclusive club. Memberships in exclusive clubs, as many companies have proven time an again, is worth money.
I think that proper understanding and adoption of Open Access will come only after we begin a dialogue where the authors and publishers are honest about the value they extract from each other. I will argue in my following posts that, in fact, the value of the brand is far less than the cost of limiting access to your work.
Barak Shahen
Monday, March 30, 2009
Orwik : The Open Science Platform
Today, I will start, what will inevitably become a new staple in my life: Blogging for Orwik. Orwik is a new company that will revolutionize the way that science is done in academia and elsewhere.
The major problem with science today is that scientists are people. Sure, we all want to do science for the pure fascination of the subject, but we also want to be respected for the science we do, we want to have the resources to do the science we want, and we want to change the world we live in with our science. How, as scientists can we accomplish any and all of these tasks?
I am sure I will say much more about the subject, but suffice it to say that unfortunately, the economics of science are not straightforward. In fact, they are really backwards. Consider this system: scientists have to write papers that are supposedly peer-reviewed and published by scientific publishing houses. Scientists are evaluated based on a somewhat obtuse formula that takes into account how many people read and cite their work by granting institutions such as NIH which translates into money for their science, and by their department which usually translates into houses, cars and college tuitions for kids.
The problem here is that the publication system is somewhat outdated. Created in the 17th century, it was never designed for the way we do science in the 21st century. We at Orwik believe that the way that science is disseminated needs a bit of a makeover. Many other scientists have also recognized that the system of science publishing is flawed.
For example, Tim Berners Lee, who was one of the founders of the web has been talking for a while about the need for change. Recently, he gave a very impassioned address at the TED conference. The link to that can be found here. There, he outlined his vision for a new web. This web, built around data, has a chance to empower new insights that we glean from the science we do. It comes from the epiphany that data in context is much more valuable than homogenized data. I agree.
In line with some of the ideas presented by Tim, we are working hard to change the way that scientists interact with each other, and how they disseminate their science. We hope that you will follow our progress here and help us add our little piece in ushering a new, open and democratic way we do science.
Barak Shahen
The major problem with science today is that scientists are people. Sure, we all want to do science for the pure fascination of the subject, but we also want to be respected for the science we do, we want to have the resources to do the science we want, and we want to change the world we live in with our science. How, as scientists can we accomplish any and all of these tasks?
I am sure I will say much more about the subject, but suffice it to say that unfortunately, the economics of science are not straightforward. In fact, they are really backwards. Consider this system: scientists have to write papers that are supposedly peer-reviewed and published by scientific publishing houses. Scientists are evaluated based on a somewhat obtuse formula that takes into account how many people read and cite their work by granting institutions such as NIH which translates into money for their science, and by their department which usually translates into houses, cars and college tuitions for kids.
The problem here is that the publication system is somewhat outdated. Created in the 17th century, it was never designed for the way we do science in the 21st century. We at Orwik believe that the way that science is disseminated needs a bit of a makeover. Many other scientists have also recognized that the system of science publishing is flawed.
For example, Tim Berners Lee, who was one of the founders of the web has been talking for a while about the need for change. Recently, he gave a very impassioned address at the TED conference. The link to that can be found here. There, he outlined his vision for a new web. This web, built around data, has a chance to empower new insights that we glean from the science we do. It comes from the epiphany that data in context is much more valuable than homogenized data. I agree.
In line with some of the ideas presented by Tim, we are working hard to change the way that scientists interact with each other, and how they disseminate their science. We hope that you will follow our progress here and help us add our little piece in ushering a new, open and democratic way we do science.
Barak Shahen
Subscribe to:
Posts (Atom)
