How did preCICE get popular?
In 2016, the German Research Foundation (DFG) published a call for proposals on Research Software Sustainability. Its goal was to build infrastructures and develop methods to make research software more sustainable and available for larger audiences. The funded software projects, in particular, should act as best-practice examples from which other researchers could learn. This research direction was radically new for DFG and the call was completely overbooked (no surprise if you ask us, find out why).
We were very happy to learn in early 2018 that preCICE was chosen as one of the few projects to fund. A few months later, we officially kicked off our three-year project preDOM – Domestication of the Coupling Library preCICE.
Some of the goals of preDOM:
- Simplify and standardize building of preCICE and provide binary packages or recipes for preCICE for a wide range of systems. (You might not remember: Back in 2018, we only provided the source code of preCICE and used SCons as the build system with quite some hidden options.)
- Set up public communication channels between developers and users.
- Develop system tests with complete coupled simulations.
- Make error messages easy to understand.
- Develop tutorials. (Yes, in 2018 we only had the 1D elastic tube as a single example.)
- … and many more.
Over the last two years, we’ve invested a lot of work. If you only got to know preCICE recently, you have no idea what the software looked like three years ago. The effort has paid off: User numbers have gone up significantly. We feel this every day when we read through the support requests.
The numbers of users that we know personally also goes up, but is certainly only the tip of the iceberg.
But which changes or events have really triggered this increase in user numbers?
One deliverable of preDOM is to answer this question, backed up with statistics as far as possible. Ideally, this helps other research software teams to prioritize their tasks. In this blog post, we present some first conclusions.
Which metric?
We have considered many different key figures: number of mailing list subscribers, number of downloads, subscribers on YouTube or also classical academic numbers (e.g. Google Scholar citations of our current reference publication). Even more number can be found in the slides of the presentation we gave at the NL-RSE 2019 conference. They all go up, but tell us little about why they go up. Single events are not traceable.
More interesting is the traffic on GitHub, since here, we can also see the short-term influence of single events. Important to know is that our users do not only go on GitHub to get preCICE, but also to access the user documentation. The specific traffic figure that GitHub offers is the number of unique visitors over the last two weeks. This figure is only visible to repository maintainers and, unfortunately, also a bit restricted: longer time periods are not available and how exactly GitHub determines unique visitors is (to our best knowledge) not documented. We’ve been monitoring the figure since the end of 2017. Interesting conclusions can be drawn by relating this figure to specific events.
What do the numbers tell us?
- The numbers have been increasing steadily. Since the end of 2017, traffic on GitHub has increased by a factor of five.
- Not so important, but funny to see: the Christmas and summer holidays are clearly visible.
- The cumulative figures (such as GitHub stars, Twitter followers, or mailing list subscribers) seem to have been increasing by an only constant rate. This could be an indicator that many users return regularly to our GitHub page. Otherwise, we would expect an exponential growth. Also, GitHub stars and Twitter is something that probably only a few of our users use.
- There has been a clearly visible increase in traffic for each new release of preCICE, especially since 2019. Of course, around the time of a new release also developers visit the site quite often. This effect, however, should be marginal as developers are typically logged into GitHub, so each one is hopefully only counted once. Instead, the increase is probably due to marketing steps we take with every release: We inform about new releases via our own channels (mailing list, website, Twitter, Research Gate, LinkedIn), but also via external channels (NADigest, CFD online). Since a few releases, we’ve also been regularly re-posted (for example by @TreeDiagram and more accounts in the Japanese CAE community or on the blog from Pointwise).
- The advertisement prior to and the preCICE Workshop itself had a huge effect on traffic.
- Interestingly, other single events, such as our preCICE minisymposia at ECCOMAS conferences (MS1, MS2) or presentations at various OpenFOAM conferences (ESI, OFW), are not always clearly visible in the traffic even though we’re convinced that they were important milestones.
So apart from the traffic figures, what criteria should we look at?
Soft success criteria
We also need to trust our gut feeling. Let’s briefly look at the most important success criteria that aren’t directly observable in the numbers, but that we’ve learned from numeruous discussions with users:
- The obvious first: The standardization of the preCICE build process and provision of binary packages for certain operating systems played an important role.
- Not as obvious is probably the following: At the moment, we guess that at least half of our users use preCICE together with the OpenFOAM adapter. So, turning the different original OpenFOAM adapters we had into the current general function object variant three years ago must have been an important milestone. Why? It allowed us to connect with an existing vivid community, the OpenFOAM community. And we tried to make things as easy as possible for them: The adapter can be built separately and works with a wide range of OpenFOAM variants, versions, and solvers. Moreover, the adapter speaks the OpenFOAM language – we use the same build system and the same configuration system as OpenFOAM. So, existing experienced OpenFOAM users don’t have to leave their comfort zone.
- On top, we’ve developed tutorial cases to let users try out preCICE right away. They’re not perfect yet, but extremely important: The wiki page on FSI with OpenFOAM and CalculiX is by far the most visited page in our documentation on GitHub.
- Lastly, the quick and patient support in our Gitter chatroom and here on Discourse is mentioned regularly by users if asked why they would recommend preCICE to others. This is great, but also dangerous, as of course, we won’t be able to keep up the support quality ourselves if user numbers increase further, and this is where we most need our community to help right now.
Recommendations
You have a great research software, but no users yet? Here’s what you can do:
- Be sure you have easy first steps: standardized and simple building and simple first test cases.
- Release in regular cycles with fixed marketing steps.
- Connect with an existing community. You develop a library? Make integration in one important software as easy as possible. You develop a standalone software? Make your input and output formats as easy as possible to integrate with existing software. Make sure your software speaks the same language as the community software you target.
- Actively request honest feedback from your users. Both bug reports and testimonials are essential for long term progress.
- Host your own workshop with tutorials and support sessions, even if you only expect a handful of guests.
Feedback
What are the most important next steps for preCICE beyond preDOM? Do you think that different reasons played an important role in the growth of preCICE? Do you think our recommendations are not generalizable? Tell us below this post.